-----============= acceptance-small: sanity-dom ============----- Thu Dec 19 23:59:25 EST 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg431-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanity-dom.*ex: No such file or directory excepting tests: === sanity-dom: start setup 23:59:37 (1734670777) === Starting client oleg431-client.virtnet: -o user_xattr,flock 192.168.204.131@tcp:/lustre /mnt/lustre2 Started clients oleg431-client.virtnet: 192.168.204.131@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg431-client.virtnet: executing check_config_client /mnt/lustre oleg431-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg431-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b5960800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8801373ff000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5960800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8801373ff000.idle_timeout=debug disable quota as required oleg431-server: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-dom: finish setup 23:59:55 (1734670795) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7026] 192.168.204.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 == sanity-dom test 1: write a file on one mount, truncate on the other, write again ========================================================== 23:59:57 (1734670797) 1+0 records in 1+0 records out 7168 bytes (7.2 kB) copied, 0.015144 s, 473 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0226236 s, 136 kB/s /mnt/lustre2/f1.sanity-dom has type file OK /mnt/lustre2/f1.sanity-dom has size 6144 OK PASS 1 (4s) == sanity-dom test 2: Write with a seek, append, read from a single mountpoint ========================================================== 00:00:01 (1734670801) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278853 s, 37.6 MB/s 1+0 records in 1+0 records out 234852 bytes (235 kB) copied, 0.00542579 s, 43.3 MB/s 0+1 records in 0+1 records out 234852 bytes (235 kB) copied, 0.0203163 s, 11.6 MB/s /mnt/lustre/f2.sanity-dom_left has type file OK /mnt/lustre/f2.sanity-dom_left has size 234852 OK PASS 2 (4s) == sanity-dom test 3: Truncate over DoM size on different nodes ========================================================== 00:00:05 (1734670805) 1+0 records in 1+0 records out 1048476 bytes (1.0 MB) copied, 0.0234627 s, 44.7 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0466527 s, 87.8 kB/s /mnt/lustre2/f3.sanity-dom has type file OK /mnt/lustre2/f3.sanity-dom has size 1049276 OK /mnt/lustre2/f3.sanity-dom has type file OK /mnt/lustre2/f3.sanity-dom has size 1049076 OK /mnt/lustre/f3.sanity-dom has type file OK /mnt/lustre/f3.sanity-dom has size 1049076 OK PASS 3 (5s) == sanity-dom test 4: DoM: glimpse doesn't produce duplicated locks ========================================================== 00:00:10 (1734670810) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0781349 s, 26.8 MB/s fail_loc=0x80000808 PASS 4 (7s) == sanity-dom test 5: DoM truncate deadlock ============== 00:00:17 (1734670817) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00666516 s, 615 kB/s multiop /mnt/lustre/f5.sanity-dom vO_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7026 multiop /mnt/lustre/f5.sanity-dom vO_Tc TMPPIPE=/tmp/multiop_open_wait_pipe.7026 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f5.sanity-dom user.job="dd.0" fail_loc=0x80001415 fail_val=5 PASS 5 (6s) == sanity-dom test 6: Race two writes, check file size === 00:00:23 (1734670823) /mnt/lustre2/f6.sanity-dom has size 500 OK PASS 6 (4s) == sanity-dom test 7: Stale pages after read-on-open ===== 00:00:28 (1734670828) 1+0 records in 1+0 records out 1000 bytes (1.0 kB) copied, 0.012812 s, 78.1 kB/s 1+0 records in 1+0 records out 1000 bytes (1.0 kB) copied, 0.0065607 s, 152 kB/s PASS 7 (4s) == sanity-dom test fsx: Dual-mount fsx with DoM files ==== 00:00:32 (1734670832) fallocate on zfs doesn't consume space fallocate not supported Chance of close/open is 1 in 50 Seed set to 833 fd 0: /mnt/lustre/ffsx.sanity-dom fd 1: /mnt/lustre2/ffsx.sanity-dom fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1734670833.539174 write 0x176d0a thru 0x1823ac (0xb6a3 bytes) 1734670833.540663 write done truncating to largest ever: 0x9fa7d 000004[0] 1734670833.587375 trunc from 0x1823ad to 0x09fa7c (0xe2930 bytes) 1734670833.601415 truncate done truncating to largest ever: 0x134620 000006[1] 1734670833.635754 trunc from 0x09fa7d to 0x13461f (0x94ba3 bytes) 1734670833.661860 truncate done 000008[1] 1734670833.694731 trunc from 0x134620 to 0x107316 (0x2d309 bytes) 1734670833.706394 truncate done 000009[0] 1734670833.719721 trunc from 0x107317 to 0x01cb00 (0xea816 bytes) 1734670833.746158 truncate done 000010[0] 1734670833.749126 mapread 0x0184ad thru 0x01cb00 (0x4654 bytes) 1734670833.749332 mmap done 1734670833.764121 memcpy done 1734670833.765090 munmap done 000014[1] 1734670833.800894 mapwrite 0x0c7bfb thru 0x0cf49c (0x78a2 bytes) 1734670833.833703 truncate done 1734670833.833902 mmap done 1734670833.915668 msync done 1734670833.916132 munmap done 000015[1] 1734670833.921429 read 0x02d0aa thru 0x038acc (0xba23 bytes) 1734670833.928566 read done 000016[1] 1734670833.958882 mapwrite 0x16bc0f thru 0x174b86 (0x8f78 bytes) 1734670833.976053 truncate done 1734670833.976290 mmap done 1734670834.035838 msync done 1734670834.036075 munmap done truncating to largest ever: 0x1bb3ca 000018[1] 1734670834.039500 trunc from 0x174b87 to 0x1bb3c9 (0x46843 bytes) 1734670834.049911 truncate done 000019[1] 1734670834.053878 mapread 0x008e41 thru 0x016249 (0xd409 bytes) 1734670834.054075 mmap done 1734670834.076566 memcpy done 1734670834.080645 munmap done 000021[1] 1734670834.104050 mapread 0x04cf79 thru 0x050213 (0x329b bytes) 1734670834.104286 mmap done 1734670834.116746 memcpy done 1734670834.116991 munmap done 000024[0] 1734670834.140595 read 0x04f02d thru 0x053950 (0x4924 bytes) 1734670834.164080 read done 000027[1] 1734670834.211298 write 0x16a0d0 thru 0x16fe37 (0x5d68 bytes) 1734670834.212098 write done 000028[1] 1734670834.252307 write 0x1a943c thru 0x1b5b70 (0xc735 bytes) 1734670834.266018 write done 000029[1] 1734670834.266363 mapwrite 0x0599a4 thru 0x05ea29 (0x5086 bytes) 1734670834.266547 mmap done 1734670834.330672 msync done 1734670834.330734 munmap done 000030[1] 1734670834.339883 mapread 0x12ca1f thru 0x131ccd (0x52af bytes) 1734670834.340129 mmap done 1734670834.349556 memcpy done 1734670834.349837 munmap done truncating to largest ever: 0x1ffa07 000031[0] 1734670834.350197 trunc from 0x1bb3ca to 0x1ffa06 (0x4463d bytes) 1734670834.389382 truncate done 000034[1] 1734670834.412609 read 0x18b196 thru 0x1949aa (0x9815 bytes) 1734670834.488616 read done 000035[1] 1734670834.492101 read 0x0b5582 thru 0x0c12bd (0xbd3c bytes) 1734670834.534383 read done 000036[1] 1734670834.544361 trunc from 0x1ffa07 to 0x162c6d (0x9cd99 bytes) 1734670834.573139 truncate done 000037[0] 1734670834.587349 mapread 0x05370e thru 0x059774 (0x6067 bytes) 1734670834.591818 mmap done 1734670834.612473 memcpy done 1734670834.613354 munmap done 000040[1] 1734670834.653769 mapread 0x01ba62 thru 0x01e58b (0x2b2a bytes) 1734670834.653950 mmap done 1734670834.682444 memcpy done 1734670834.682680 munmap done 000041[0] 1734670834.683036 mapread 0x0fc45f thru 0x10c1c7 (0xfd69 bytes) 1734670834.695119 mmap done 1734670834.770678 memcpy done 1734670834.770870 munmap done 000048[0] 1734670834.775100 write 0x1c8453 thru 0x1d7efe (0xfaac bytes) 1734670834.792716 write done 000049[1] 1734670834.793061 mapwrite 0x101161 thru 0x107b2d (0x69cd bytes) 1734670834.804899 mmap done 1734670834.975085 msync done 1734670834.975179 munmap done 000050[1] 1734670834.975487 read 0x077729 thru 0x07f498 (0x7d70 bytes) 1734670835.020002 read done 000051[1] 1734670835.042119 trunc from 0x1d7eff to 0x0ff682 (0xd887c bytes) 1734670835.081995 truncate done 000054[0] 1734670835.105442 write 0x121f51 thru 0x128279 (0x6329 bytes) 1734670835.117442 write done 000056[0] 1734670835.146428 read 0x07d5da thru 0x08b9c1 (0xe3e8 bytes) 1734670835.195216 read done 000057[1] 1734670835.195597 mapread 0x04053a thru 0x04b1e8 (0xacaf bytes) 1734670835.201584 mmap done 1734670835.246880 memcpy done 1734670835.247254 munmap done 000060[0] 1734670835.272427 trunc from 0x12827a to 0x17d1fe (0x54f85 bytes) 1734670835.297935 truncate done 000063[1] 1734670835.324612 trunc from 0x17d1ff to 0x11a82e (0x629d0 bytes) 1734670835.369671 truncate done 000064[0] 1734670835.382653 write 0x0f19a3 thru 0x0f2ff2 (0x1650 bytes) 1734670835.397000 write done 000065[0] 1734670835.411364 read 0x0fde64 thru 0x1042cc (0x6469 bytes) 1734670835.461044 read done 000066[1] 1734670835.472603 read 0x10c88b thru 0x110c28 (0x439e bytes) 1734670835.489246 read done 000069[1] 1734670835.501954 mapwrite 0x00eae6 thru 0x0154e4 (0x69ff bytes) 1734670835.507112 mmap done 1734670835.601383 msync done 1734670835.601817 munmap done 000070[1] 1734670835.613308 trunc from 0x11a82f to 0x1d81fe (0xbd9d0 bytes) 1734670835.638589 truncate done 000071[1] 1734670835.643125 mapwrite 0x09b45c thru 0x0a0bd6 (0x577b bytes) 1734670835.643356 mmap done 1734670835.684837 msync done 1734670835.684891 munmap done 000073[0] 1734670835.722501 trunc from 0x1d81ff to 0x09af43 (0x13d2bb bytes) 1734670835.763117 truncate done 000074[1] 1734670835.779837 read 0x02abdb thru 0x02ebf7 (0x401d bytes) 1734670835.807189 read done 000075[0] 1734670835.812793 write 0x03b9ef thru 0x04156f (0x5b81 bytes) 1734670835.831595 write done 000078[1] 1734670835.860959 read 0x0293b3 thru 0x0390a4 (0xfcf2 bytes) 1734670835.887508 read done 000079[0] 1734670835.901704 read 0x04cedc thru 0x05c55d (0xf682 bytes) 1734670835.925753 read done 000082[0] 1734670835.936031 mapread 0x0645a3 thru 0x07096c (0xc3ca bytes) 1734670835.936910 mmap done 1734670835.936992 memcpy done 1734670835.937169 munmap done 000083[1] 1734670835.938894 read 0x0524f2 thru 0x052b34 (0x643 bytes) 1734670835.939004 read done 000088[0] 1734670835.977972 mapread 0x08ea4e thru 0x08ec3c (0x1ef bytes) 1734670835.978204 mmap done 1734670835.978252 memcpy done 1734670835.979196 munmap done 000094[1] 1734670836.029090 read 0x074d7e thru 0x07a489 (0x570c bytes) 1734670836.029225 read done 000095[1] 1734670836.029557 write 0x0c6c74 thru 0x0c8b9d (0x1f2a bytes) 1734670836.047834 write done 000097[0] 1734670836.084289 mapwrite 0x17bf16 thru 0x185d4c (0x9e37 bytes) 1734670836.119154 truncate done 1734670836.119334 mmap done 1734670836.268068 msync done 1734670836.268289 munmap done 000100[0] 1734670836.328855 trunc from 0x185d4d to 0x04b205 (0x13ab47 bytes) 1734670836.341820 truncate done 000103[1] 1734670836.379586 trunc from 0x04b206 to 0x1d017c (0x184f77 bytes) 1734670836.405791 truncate done 000104[1] 1734670836.410754 write 0x1c198a thru 0x1c8522 (0x6b99 bytes) 1734670836.437261 write done 000106[1] 1734670836.454116 trunc from 0x1d017d to 0x047659 (0x188b23 bytes) 1734670836.471421 truncate done 000109[1] 1734670836.568099 mapwrite 0x0ee6d6 thru 0x0f2e42 (0x476d bytes) 1734670836.581139 truncate done 1734670836.581759 mmap done 1734670836.641429 msync done 1734670836.641486 munmap done 000110[1] 1734670836.655521 mapwrite 0x102223 thru 0x10618c (0x3f6a bytes) 1734670836.665388 truncate done 1734670836.665577 mmap done 1734670836.754773 msync done 1734670836.755024 munmap done 000113[1] 1734670836.804319 mapwrite 0x1cae68 thru 0x1cb247 (0x3e0 bytes) 1734670836.816834 truncate done 1734670836.817302 mmap done 1734670836.869583 msync done 1734670836.870263 munmap done 000115[0] 1734670836.874342 trunc from 0x1cb248 to 0x08ff6b (0x13b2dc bytes) 1734670836.907133 truncate done 000117[0] 1734670836.927142 trunc from 0x08ff6c to 0x08fb4d (0x41e bytes) 1734670836.943439 truncate done 000121[1] 1734670836.991042 mapwrite 0x0a659d thru 0x0af85d (0x92c1 bytes) 1734670837.005351 truncate done 1734670837.005553 mmap done 1734670837.115955 msync done 1734670837.117586 munmap done 000123[1] 1734670837.162259 mapread 0x0441e1 thru 0x051e05 (0xdc25 bytes) 1734670837.162527 mmap done 1734670837.241790 memcpy done 1734670837.243236 munmap done 000125[0] 1734670837.259883 trunc from 0x0af85e to 0x08ce06 (0x22a57 bytes) 1734670837.332560 truncate done 000126[0] 1734670837.354573 write 0x18a07b thru 0x18da52 (0x39d8 bytes) 1734670837.355383 write done 000127[0] 1734670837.361445 mapwrite 0x10e50e thru 0x1101cc (0x1cbf bytes) 1734670837.361659 mmap done 1734670837.456211 msync done 1734670837.456452 munmap done 000128[0] 1734670837.475135 read 0x1726c1 thru 0x174332 (0x1c72 bytes) 1734670837.498063 read done 000129[1] 1734670837.498490 read 0x045e17 thru 0x04ea59 (0x8c43 bytes) 1734670837.533908 read done 000130[1] 1734670837.564811 read 0x14b77a thru 0x1563a4 (0xac2b bytes) 1734670837.601777 read done 000131[1] 1734670837.602183 mapwrite 0x1cd1a4 thru 0x1d79c2 (0xa81f bytes) 1734670837.636936 truncate done 1734670837.637185 mmap done 1734670837.718998 msync done 1734670837.719894 munmap done 000133[0] 1734670837.754745 read 0x15c331 thru 0x163af1 (0x77c1 bytes) 1734670837.829083 read done 000135[0] 1734670837.832982 mapwrite 0x037dc0 thru 0x0451b0 (0xd3f1 bytes) 1734670837.838700 mmap done 1734670837.937815 msync done 1734670837.938032 munmap done 000136[1] 1734670837.938486 mapwrite 0x04e680 thru 0x05e61f (0xffa0 bytes) 1734670837.943388 mmap done 1734670838.080181 msync done 1734670838.080284 munmap done 000137[0] 1734670838.088585 mapread 0x106101 thru 0x111a7d (0xb97d bytes) 1734670838.093973 mmap done 1734670838.094108 memcpy done 1734670838.094321 munmap done 000138 1734670838.125735 [1]close/open 1734670838.129654 close done 1734670838.137760 open done 000139[1] 1734670838.140384 mapread 0x1339f6 thru 0x136b1c (0x3127 bytes) 1734670838.143503 mmap done 1734670838.172489 memcpy done 1734670838.173495 munmap done 000142[0] 1734670838.221172 trunc from 0x1d79c3 to 0x17e904 (0x590be bytes) 1734670838.250959 truncate done 000145[0] 1734670838.273048 mapwrite 0x0bb469 thru 0x0c8339 (0xced1 bytes) 1734670838.273358 mmap done 1734670838.406916 msync done 1734670838.407694 munmap done 000146[1] 1734670838.436234 read 0x06220f thru 0x066c7b (0x4a6d bytes) 1734670838.455222 read done 000152[1] 1734670838.518893 mapread 0x03c14e thru 0x040bf9 (0x4aac bytes) 1734670838.524927 mmap done 1734670838.530254 memcpy done 1734670838.530657 munmap done 000153[0] 1734670838.543556 mapwrite 0x19b083 thru 0x1a8011 (0xcf8f bytes) 1734670838.565164 truncate done 1734670838.565367 mmap done 1734670838.676613 msync done 1734670838.676956 munmap done 000154[1] 1734670838.694334 read 0x0b66a7 thru 0x0be019 (0x7973 bytes) 1734670838.718523 read done 000158[0] 1734670838.745944 write 0x0f5f70 thru 0x0fe2e1 (0x8372 bytes) 1734670838.763388 write done 000159[1] 1734670838.776061 write 0x1cecce thru 0x1d167c (0x29af bytes) 1734670838.785283 write done 000162[1] 1734670838.814166 read 0x019d78 thru 0x025177 (0xb400 bytes) 1734670838.864134 read done 000163[1] 1734670838.879612 mapwrite 0x1191aa thru 0x122227 (0x907e bytes) 1734670838.880364 mmap done 1734670838.934174 msync done 1734670838.934254 munmap done 000164[0] 1734670838.934614 mapread 0x0e6580 thru 0x0f43e2 (0xde63 bytes) 1734670838.944060 mmap done 1734670839.034334 memcpy done 1734670839.034406 munmap done 000165[0] 1734670839.035713 trunc from 0x1d167d to 0x1fd9b7 (0x2c33b bytes) 1734670839.092681 truncate done 000168[1] 1734670839.121481 trunc from 0x1fd9b8 to 0x03360d (0x1ca3aa bytes) 1734670839.153910 truncate done 000169[1] 1734670839.168247 write 0x090a4e thru 0x09b671 (0xac24 bytes) 1734670839.169267 write done 000170[0] 1734670839.172729 read 0x029fe0 thru 0x02b35d (0x137e bytes) 1734670839.201508 read done 000172[0] 1734670839.214856 read 0x036167 thru 0x03d4a6 (0x7340 bytes) 1734670839.215328 read done 000174[0] 1734670839.225579 trunc from 0x09b672 to 0x01f3f6 (0x7c27b bytes) 1734670839.248308 truncate done 000176[1] 1734670839.291333 read 0x006464 thru 0x0157c4 (0xf361 bytes) 1734670839.308087 read done 000177[0] 1734670839.311718 read 0x01efef thru 0x01f3f6 (0x408 bytes) 1734670839.320749 read done 000179[1] 1734670839.338924 mapwrite 0x1fda7b thru 0x1fffff (0x2585 bytes) 1734670839.363821 truncate done 1734670839.364000 mmap done 1734670839.494840 msync done 1734670839.495094 munmap done 000180[1] 1734670839.519711 write 0x0af4d0 thru 0x0bdb3a (0xe66b bytes) 1734670839.520410 write done 000181[1] 1734670839.551164 read 0x1334ae thru 0x1360c0 (0x2c13 bytes) 1734670839.551313 read done 000183[1] 1734670839.633442 trunc from 0x200000 to 0x02482b (0x1db7d4 bytes) 1734670839.680193 truncate done 000185[0] 1734670839.685395 read 0x00f530 thru 0x01d56e (0xe03f bytes) 1734670839.705590 read done 000189[0] 1734670839.747557 mapread 0x0187fc thru 0x021166 (0x896b bytes) 1734670839.753495 mmap done 1734670839.753605 memcpy done 1734670839.753944 munmap done 000190[1] 1734670839.754340 mapread 0x001f39 thru 0x00acb3 (0x8d7b bytes) 1734670839.754503 mmap done 1734670839.763360 memcpy done 1734670839.763443 munmap done 000191[1] 1734670839.763838 write 0x15329c thru 0x15b368 (0x80cd bytes) 1734670839.764666 write done 000192[1] 1734670839.764928 read 0x0945f8 thru 0x09b360 (0x6d69 bytes) 1734670839.776473 read done 000197[0] 1734670839.807054 mapwrite 0x0619ac thru 0x06b857 (0x9eac bytes) 1734670839.818567 mmap done 1734670840.052158 msync done 1734670840.052387 munmap done 000198[1] 1734670840.064243 trunc from 0x15b369 to 0x18968d (0x2e325 bytes) 1734670840.081540 truncate done 000200[1] 1734670840.102718 write 0x154bb8 thru 0x1562c7 (0x1710 bytes) 1734670840.103138 write done 000201[0] 1734670840.118699 mapread 0x09a8be thru 0x09df79 (0x36bc bytes) 1734670840.128864 mmap done 1734670840.181370 memcpy done 1734670840.181851 munmap done 000203[1] 1734670840.206789 read 0x06846e thru 0x07675a (0xe2ed bytes) 1734670840.232757 read done 000204[1] 1734670840.233132 mapread 0x18467c thru 0x185ced (0x1672 bytes) 1734670840.233299 mmap done 1734670840.233335 memcpy done 1734670840.233540 munmap done 000205[1] 1734670840.234789 trunc from 0x18968e to 0x1d294e (0x492c1 bytes) 1734670840.260890 truncate done 000207[1] 1734670840.263564 mapwrite 0x058c02 thru 0x06411f (0xb51e bytes) 1734670840.263906 mmap done 1734670840.362816 msync done 1734670840.363082 munmap done 000209[1] 1734670840.384112 trunc from 0x1d294f to 0x0fe611 (0xd433d bytes) 1734670840.397411 truncate done 000210[1] 1734670840.416766 trunc from 0x0fe612 to 0x1d022c (0xd1c1b bytes) 1734670840.427800 truncate done 000212[0] 1734670840.431988 read 0x0558af thru 0x060a57 (0xb1a9 bytes) 1734670840.464062 read done 000213[1] 1734670840.467734 read 0x1a2656 thru 0x1a883c (0x61e7 bytes) 1734670840.491476 read done 000215[1] 1734670840.520040 mapread 0x0c1b80 thru 0x0c75ae (0x5a2f bytes) 1734670840.520310 mmap done 1734670840.520392 memcpy done 1734670840.520612 munmap done 000216[0] 1734670840.521602 write 0x1281f8 thru 0x12c0a0 (0x3ea9 bytes) 1734670840.549989 write done 000218[1] 1734670840.561182 mapwrite 0x184a9c thru 0x191970 (0xced5 bytes) 1734670840.566152 mmap done 1734670840.714130 msync done 1734670840.714351 munmap done 000219[1] 1734670840.724932 write 0x098fc1 thru 0x099bac (0xbec bytes) 1734670840.747469 write done 000220[1] 1734670840.762096 write 0x1701f4 thru 0x17e94e (0xe75b bytes) 1734670840.769115 write done 000222[1] 1734670840.792832 mapwrite 0x08089a thru 0x086e7b (0x65e2 bytes) 1734670840.793136 mmap done 1734670840.852253 msync done 1734670840.852732 munmap done 000223[0] 1734670840.872098 trunc from 0x1d022d to 0x06d2de (0x162f4e bytes) 1734670840.920846 truncate done 000226[1] 1734670840.959247 mapread 0x031a83 thru 0x03ed40 (0xd2be bytes) 1734670840.970702 mmap done 1734670841.034823 memcpy done 1734670841.035130 munmap done 000227[1] 1734670841.039378 read 0x067739 thru 0x06d2de (0x5ba6 bytes) 1734670841.050452 read done 000233[1] 1734670841.093794 mapwrite 0x180383 thru 0x180479 (0xf7 bytes) 1734670841.143610 truncate done 1734670841.145162 mmap done 1734670841.254430 msync done 1734670841.254877 munmap done 000234[1] 1734670841.259037 read 0x04d17c thru 0x052c69 (0x5aee bytes) 1734670841.272188 read done 000235[1] 1734670841.273436 mapwrite 0x131d7d thru 0x135365 (0x35e9 bytes) 1734670841.273662 mmap done 1734670841.312907 msync done 1734670841.314423 munmap done 000237[1] 1734670841.350120 read 0x152d99 thru 0x154a3e (0x1ca6 bytes) 1734670841.350288 read done 000238[1] 1734670841.350605 mapwrite 0x1de6c8 thru 0x1e06f3 (0x202c bytes) 1734670841.365015 truncate done 1734670841.365292 mmap done 1734670841.409511 msync done 1734670841.409791 munmap done 000244[1] 1734670841.470906 mapwrite 0x104e0c thru 0x10e2cf (0x94c4 bytes) 1734670841.471274 mmap done 1734670841.522027 msync done 1734670841.522503 munmap done 000245 1734670841.522936 [0]close/open 1734670841.527559 close done 1734670841.538782 open done 000246[1] 1734670841.587560 read 0x1422da thru 0x14dff4 (0xbd1b bytes) 1734670841.587755 read done 000248[0] 1734670841.611862 mapwrite 0x07a23f thru 0x080a49 (0x680b bytes) 1734670841.626468 mmap done 1734670841.788045 msync done 1734670841.788445 munmap done 000250[1] 1734670841.808337 mapread 0x18d822 thru 0x19735b (0x9b3a bytes) 1734670841.813128 mmap done 1734670841.813241 memcpy done 1734670841.813484 munmap done 000255[1] 1734670841.863688 mapwrite 0x02d568 thru 0x02e3be (0xe57 bytes) 1734670841.868613 mmap done 1734670841.931899 msync done 1734670841.932033 munmap done 000256[0] 1734670841.932391 read 0x027d0a thru 0x031cf2 (0x9fe9 bytes) 1734670841.954378 read done 000257[1] 1734670841.961756 mapread 0x148330 thru 0x156d62 (0xea33 bytes) 1734670841.961991 mmap done 1734670841.963178 memcpy done 1734670841.963317 munmap done 000260[0] 1734670841.966958 write 0x036748 thru 0x045300 (0xebb9 bytes) 1734670842.000410 write done 000261[1] 1734670842.025013 trunc from 0x1e06f4 to 0x008c11 (0x1d7ae2 bytes) 1734670842.064930 truncate done 000262[0] 1734670842.071105 mapread 0x006b5b thru 0x008c11 (0x20b7 bytes) 1734670842.081815 mmap done 1734670842.101047 memcpy done 1734670842.101955 munmap done 000266[0] 1734670842.144991 write 0x113592 thru 0x11d7a8 (0xa217 bytes) 1734670842.154128 write done 000270[1] 1734670842.190937 mapread 0x02d49c thru 0x02e4a6 (0x100b bytes) 1734670842.203597 mmap done 1734670842.239053 memcpy done 1734670842.239885 munmap done 000273[1] 1734670842.265594 mapread 0x0c096b thru 0x0cd982 (0xd018 bytes) 1734670842.273120 mmap done 1734670842.431148 memcpy done 1734670842.431912 munmap done 000274[0] 1734670842.444284 read 0x0acbb7 thru 0x0b7e88 (0xb2d2 bytes) 1734670842.459486 read done 000275[1] 1734670842.461031 mapread 0x0e88a4 thru 0x0f5054 (0xc7b1 bytes) 1734670842.467125 mmap done 1734670842.673017 memcpy done 1734670842.678508 munmap done 000277[0] 1734670842.680293 trunc from 0x11d7a9 to 0x13c316 (0x1eb6e bytes) 1734670842.715489 truncate done 000278[1] 1734670842.730013 mapread 0x029f9b thru 0x02b7d0 (0x1836 bytes) 1734670842.737840 mmap done 1734670842.773589 memcpy done 1734670842.774728 munmap done 000282[0] 1734670842.822965 write 0x088899 thru 0x0984b3 (0xfc1b bytes) 1734670842.842409 write done 000283[0] 1734670842.858015 write 0x01ac2a thru 0x01b92b (0xd02 bytes) 1734670842.866696 write done 000284[0] 1734670842.867546 mapwrite 0x0eed40 thru 0x0f2430 (0x36f1 bytes) 1734670842.867765 mmap done 1734670842.933169 msync done 1734670842.933553 munmap done 000286[1] 1734670842.957755 mapwrite 0x172328 thru 0x178ffd (0x6cd6 bytes) 1734670843.006486 truncate done 1734670843.007313 mmap done 1734670843.129245 msync done 1734670843.129329 munmap done 000287[0] 1734670843.149104 read 0x0c662d thru 0x0ca46a (0x3e3e bytes) 1734670843.179318 read done 000289[1] 1734670843.201173 trunc from 0x178ffe to 0x0a622e (0xd2dcf bytes) 1734670843.231126 truncate done 000291[0] 1734670843.280107 write 0x1e8f34 thru 0x1f86e3 (0xf7b0 bytes) 1734670843.298296 write done 000292[0] 1734670843.327918 mapread 0x010137 thru 0x01f41e (0xf2e8 bytes) 1734670843.337041 mmap done 1734670843.483249 memcpy done 1734670843.483797 munmap done 000294[1] 1734670843.484711 trunc from 0x1f86e4 to 0x0ddffc (0x11a6e7 bytes) 1734670843.551044 truncate done 000295[0] 1734670843.553976 mapread 0x034131 thru 0x043461 (0xf331 bytes) 1734670843.561973 mmap done 1734670843.623148 memcpy done 1734670843.623531 munmap done 000296[0] 1734670843.626215 mapread 0x0a05a3 thru 0x0ab23e (0xac9c bytes) 1734670843.634069 mmap done 1734670843.711785 memcpy done 1734670843.712403 munmap done 000298[1] 1734670843.727503 trunc from 0x0ddffd to 0x1a55c7 (0xc75cb bytes) 1734670843.747964 truncate done 000301[1] 1734670843.777843 read 0x124611 thru 0x1293e2 (0x4dd2 bytes) 1734670843.800123 read done 000303[0] 1734670843.826034 write 0x184873 thru 0x192de4 (0xe572 bytes) 1734670843.846518 write done 000304[1] 1734670843.864579 mapread 0x12cfeb thru 0x137f6e (0xaf84 bytes) 1734670843.872112 mmap done 1734670843.952532 memcpy done 1734670843.952971 munmap done 000305[1] 1734670843.953345 trunc from 0x1a55c8 to 0x1095c1 (0x9c006 bytes) 1734670843.982724 truncate done 000308[1] 1734670844.039854 write 0x084e9e thru 0x085a99 (0xbfc bytes) 1734670844.040308 write done 000310[0] 1734670844.060220 read 0x044abd thru 0x04e054 (0x9598 bytes) 1734670844.098045 read done 000312[0] 1734670844.120881 mapwrite 0x1bc923 thru 0x1c2985 (0x6063 bytes) 1734670844.168470 truncate done 1734670844.168731 mmap done 1734670844.280269 msync done 1734670844.282061 munmap done 000315[0] 1734670844.325836 mapwrite 0x191aec thru 0x19b4b8 (0x99cd bytes) 1734670844.326122 mmap done 1734670844.376776 msync done 1734670844.376876 munmap done 000316[1] 1734670844.394193 write 0x1e8593 thru 0x1f0e65 (0x88d3 bytes) 1734670844.403785 write done 000319[0] 1734670844.435759 mapwrite 0x12fe65 thru 0x136861 (0x69fd bytes) 1734670844.441835 mmap done 1734670844.541897 msync done 1734670844.542183 munmap done 000323[1] 1734670844.594738 mapwrite 0x0c88fd thru 0x0d1cfb (0x93ff bytes) 1734670844.605881 mmap done 1734670844.774241 msync done 1734670844.774869 munmap done 000326[0] 1734670844.812784 mapread 0x0ae166 thru 0x0b21ab (0x4046 bytes) 1734670844.821052 mmap done 1734670844.850239 memcpy done 1734670844.851931 munmap done 000329[0] 1734670844.884763 read 0x1ebc0a thru 0x1f0e65 (0x525c bytes) 1734670844.885460 read done 000331[1] 1734670844.899939 mapwrite 0x19390c thru 0x194539 (0xc2e bytes) 1734670844.904143 mmap done 1734670844.973838 msync done 1734670844.973910 munmap done 000333[0] 1734670844.975177 mapwrite 0x01446c thru 0x01af20 (0x6ab5 bytes) 1734670844.981611 mmap done 1734670845.051317 msync done 1734670845.051396 munmap done 000336[1] 1734670845.095725 write 0x047a9a thru 0x04f5da (0x7b41 bytes) 1734670845.112034 write done 000338[0] 1734670845.126260 write 0x17dea2 thru 0x189800 (0xb95f bytes) 1734670845.143017 write done 000339[0] 1734670845.152703 mapwrite 0x131438 thru 0x1356c1 (0x428a bytes) 1734670845.159418 mmap done 1734670845.222848 msync done 1734670845.222930 munmap done 000341[0] 1734670845.243713 trunc from 0x1f0e66 to 0x0636d4 (0x18d791 bytes) 1734670845.269395 truncate done 000345[1] 1734670845.307440 mapread 0x02800d thru 0x033d42 (0xbd36 bytes) 1734670845.314287 mmap done 1734670845.368380 memcpy done 1734670845.368783 munmap done 000347[0] 1734670845.372112 mapwrite 0x1f504a thru 0x1ff129 (0xa0e0 bytes) 1734670845.399801 truncate done 1734670845.400014 mmap done 1734670845.531541 msync done 1734670845.532560 munmap done 000350[1] 1734670845.599924 read 0x1322f6 thru 0x136fd9 (0x4ce4 bytes) 1734670845.643617 read done 000353[0] 1734670845.666126 trunc from 0x1ff12a to 0x077e36 (0x1872f3 bytes) 1734670845.690365 truncate done 000354[0] 1734670845.708280 mapread 0x0583cb thru 0x0682c8 (0xfefe bytes) 1734670845.708553 mmap done 1734670845.770801 memcpy done 1734670845.771673 munmap done 000358[1] 1734670845.790586 read 0x0116b1 thru 0x0127a7 (0x10f7 bytes) 1734670845.815393 read done 000359[1] 1734670845.822868 mapwrite 0x070098 thru 0x079abe (0x9a27 bytes) 1734670845.859090 truncate done 1734670845.859305 mmap done 1734670845.940566 msync done 1734670845.940841 munmap done 000360[0] 1734670845.948797 mapread 0x047a5c thru 0x04aa19 (0x2fbe bytes) 1734670845.960393 mmap done 1734670845.978431 memcpy done 1734670845.978500 munmap done 000361[0] 1734670845.987921 mapread 0x01a969 thru 0x028b4b (0xe1e3 bytes) 1734670845.993279 mmap done 1734670846.042919 memcpy done 1734670846.043013 munmap done 000365[0] 1734670846.066025 mapwrite 0x141c04 thru 0x146629 (0x4a26 bytes) 1734670846.091959 truncate done 1734670846.092164 mmap done 1734670846.162179 msync done 1734670846.162399 munmap done 000366[1] 1734670846.165025 mapread 0x118659 thru 0x11f8a8 (0x7250 bytes) 1734670846.172433 mmap done 1734670846.206775 memcpy done 1734670846.207135 munmap done 000367[0] 1734670846.223503 trunc from 0x14662a to 0x1a84af (0x61e86 bytes) 1734670846.241746 truncate done 000368[0] 1734670846.245491 trunc from 0x1a84b0 to 0x19ef92 (0x951d bytes) 1734670846.252225 truncate done 000370[0] 1734670846.269884 trunc from 0x19ef93 to 0x17e187 (0x20e0b bytes) 1734670846.280873 truncate done 000371[1] 1734670846.285414 read 0x12c003 thru 0x12c8fd (0x8fb bytes) 1734670846.304386 read done 000372[0] 1734670846.306342 trunc from 0x17e188 to 0x0572bd (0x126eca bytes) 1734670846.318012 truncate done 000373[1] 1734670846.320163 mapread 0x022f83 thru 0x02fc9b (0xcd19 bytes) 1734670846.324556 mmap done 1734670846.356208 memcpy done 1734670846.356322 munmap done 000374[1] 1734670846.360854 trunc from 0x0572be to 0x1695dc (0x11231f bytes) 1734670846.396809 truncate done 000376[1] 1734670846.418577 trunc from 0x1695dd to 0x06207b (0x107561 bytes) 1734670846.436133 truncate done 000380[1] 1734670846.480757 read 0x021d0b thru 0x02960f (0x7905 bytes) 1734670846.497690 read done 000385[0] 1734670846.564765 mapwrite 0x1c8718 thru 0x1d44ce (0xbdb7 bytes) 1734670846.613263 truncate done 1734670846.613500 mmap done 1734670846.830993 msync done 1734670846.831297 munmap done 000387[0] 1734670846.851875 write 0x0209c6 thru 0x02d542 (0xcb7d bytes) 1734670846.861555 write done 000389[1] 1734670846.886107 write 0x0c4da0 thru 0x0c625a (0x14bb bytes) 1734670846.898173 write done 000391[1] 1734670846.921516 mapwrite 0x092e1a thru 0x09a1a0 (0x7387 bytes) 1734670846.926047 mmap done 1734670846.979846 msync done 1734670846.980088 munmap done 000392[1] 1734670846.991735 mapread 0x00fa1b thru 0x016a10 (0x6ff6 bytes) 1734670846.997560 mmap done 1734670847.025525 memcpy done 1734670847.026173 munmap done 000397[1] 1734670847.076133 read 0x1d3c02 thru 0x1d44ce (0x8cd bytes) 1734670847.115640 read done 000398[0] 1734670847.115988 read 0x18c0d8 thru 0x18d436 (0x135f bytes) 1734670847.137477 read done 000399[1] 1734670847.147897 write 0x13d751 thru 0x1478e4 (0xa194 bytes) 1734670847.164097 write done 000402[1] 1734670847.164890 mapread 0x09add1 thru 0x0a5cff (0xaf2f bytes) 1734670847.165043 mmap done 1734670847.165141 memcpy done 1734670847.165504 munmap done 000404[0] 1734670847.177988 mapread 0x0f990d thru 0x0ffe98 (0x658c bytes) 1734670847.183036 mmap done 1734670847.263521 memcpy done 1734670847.265475 munmap done 000406[1] 1734670847.290678 mapwrite 0x18ac93 thru 0x18f4eb (0x4859 bytes) 1734670847.291595 mmap done 1734670847.407317 msync done 1734670847.407379 munmap done 000407[0] 1734670847.419169 mapread 0x000108 thru 0x00f1e1 (0xf0da bytes) 1734670847.431162 mmap done 1734670847.431290 memcpy done 1734670847.431319 munmap done 000413[1] 1734670847.469151 trunc from 0x1d44cf to 0x1b13c4 (0x2310a bytes) 1734670847.497049 truncate done 000415[1] 1734670847.529024 trunc from 0x1b13c5 to 0x007162 (0x1aa262 bytes) 1734670847.542557 truncate done 000418[1] 1734670847.561221 trunc from 0x007163 to 0x13f40d (0x1382ab bytes) 1734670847.574341 truncate done 000422[1] 1734670847.616204 write 0x15b042 thru 0x167fea (0xcfa9 bytes) 1734670847.617568 write done 000425[1] 1734670847.618775 write 0x0cbb78 thru 0x0d1e2a (0x62b3 bytes) 1734670847.626491 write done 000426[0] 1734670847.637469 mapread 0x11e4c5 thru 0x127012 (0x8b4e bytes) 1734670847.643990 mmap done 1734670847.702980 memcpy done 1734670847.703254 munmap done 000427[1] 1734670847.715154 read 0x1233d0 thru 0x12f230 (0xbe61 bytes) 1734670847.746139 read done 000428[1] 1734670847.746554 trunc from 0x167feb to 0x1a9201 (0x41217 bytes) 1734670847.792336 truncate done 000429[1] 1734670847.823064 write 0x0d060c thru 0x0d2d95 (0x278a bytes) 1734670847.826906 write done 000430[0] 1734670847.856914 mapwrite 0x06530e thru 0x068253 (0x2f46 bytes) 1734670847.866572 mmap done 1734670847.959965 msync done 1734670847.960064 munmap done 000433[0] 1734670848.014431 trunc from 0x1a9202 to 0x0a1161 (0x1080a0 bytes) 1734670848.031279 truncate done 000434[1] 1734670848.052777 trunc from 0x0a1162 to 0x03d5ef (0x63b72 bytes) 1734670848.068043 truncate done 000435[1] 1734670848.080840 mapread 0x01bce9 thru 0x020667 (0x497f bytes) 1734670848.081120 mmap done 1734670848.111658 memcpy done 1734670848.112781 munmap done 000437[0] 1734670848.144675 write 0x02b99d thru 0x034cd0 (0x9334 bytes) 1734670848.165708 write done 000439[1] 1734670848.202778 mapwrite 0x1a6069 thru 0x1a6177 (0x10f bytes) 1734670848.261870 truncate done 1734670848.262484 mmap done 1734670848.395008 msync done 1734670848.395090 munmap done 000440[0] 1734670848.439209 read 0x09bf77 thru 0x0a657d (0xa607 bytes) 1734670848.541980 read done 000441[0] 1734670848.551005 read 0x05f114 thru 0x06e15b (0xf048 bytes) 1734670848.576048 read done 000444[1] 1734670848.592905 read 0x0987d1 thru 0x0a3f2a (0xb75a bytes) 1734670848.620101 read done 000445[0] 1734670848.639909 read 0x09c9c9 thru 0x0a4096 (0x76ce bytes) 1734670848.654052 read done 000447[0] 1734670848.668147 mapread 0x1995c1 thru 0x19b915 (0x2355 bytes) 1734670848.672546 mmap done 1734670848.720905 memcpy done 1734670848.721155 munmap done 000453[1] 1734670848.728824 read 0x12c5db thru 0x12e604 (0x202a bytes) 1734670848.747958 read done 000454[0] 1734670848.749408 write 0x1a7891 thru 0x1a85ba (0xd2a bytes) 1734670848.759573 write done 000455[0] 1734670848.759982 read 0x05d9e2 thru 0x06d48f (0xfaae bytes) 1734670848.760131 read done 000459[1] 1734670848.778611 mapwrite 0x1ab9e3 thru 0x1b8a67 (0xd085 bytes) 1734670848.819980 truncate done 1734670848.820409 mmap done 1734670848.892853 msync done 1734670848.892997 munmap done 000461[1] 1734670848.937569 write 0x14f649 thru 0x150e46 (0x17fe bytes) 1734670848.951199 write done 000466[0] 1734670848.997112 write 0x10b839 thru 0x11274d (0x6f15 bytes) 1734670849.023137 write done 000469[1] 1734670849.113337 read 0x02b09c thru 0x02b1c8 (0x12d bytes) 1734670849.134519 read done 000474[0] 1734670849.212840 mapread 0x1a9a2e thru 0x1b8a67 (0xf03a bytes) 1734670849.220433 mmap done 1734670849.274348 memcpy done 1734670849.274560 munmap done 000476[1] 1734670849.296166 mapread 0x07238c thru 0x07ea02 (0xc677 bytes) 1734670849.301432 mmap done 1734670849.400873 memcpy done 1734670849.401096 munmap done 000479[0] 1734670849.434118 write 0x0e694e thru 0x0f4451 (0xdb04 bytes) 1734670849.467303 write done 000480[0] 1734670849.467614 read 0x0ee9b3 thru 0x0fc108 (0xd756 bytes) 1734670849.467730 read done 000484[1] 1734670849.482065 mapread 0x07d8ae thru 0x082c29 (0x537c bytes) 1734670849.488156 mmap done 1734670849.523857 memcpy done 1734670849.524078 munmap done 000487[1] 1734670849.529474 trunc from 0x1b8a68 to 0x02b9ec (0x18d07b bytes) 1734670849.567828 truncate done 000489[0] 1734670849.601758 trunc from 0x02b9ed to 0x19049a (0x164aae bytes) 1734670849.623789 truncate done 000492[1] 1734670849.666805 read 0x01bf18 thru 0x0203e7 (0x44d0 bytes) 1734670849.687079 read done 000493[0] 1734670849.697540 write 0x199ecb thru 0x19a0f1 (0x227 bytes) 1734670849.697982 write done 000494[1] 1734670849.702608 mapwrite 0x04d87c thru 0x05ab91 (0xd316 bytes) 1734670849.710216 mmap done 1734670849.788957 msync done 1734670849.789161 munmap done 000497[0] 1734670849.827014 write 0x1b1672 thru 0x1c115e (0xfaed bytes) 1734670849.828301 write done 000498[1] 1734670849.851403 read 0x10362e thru 0x10b5ea (0x7fbd bytes) 1734670849.926187 read done 000499[0] 1734670849.926546 mapread 0x0b5678 thru 0x0bec25 (0x95ae bytes) 1734670849.936439 mmap done 1734670849.979179 memcpy done 1734670849.981062 munmap done 000500[0] 1734670849.982553 mapwrite 0x0c7a5f thru 0x0d5266 (0xd808 bytes) 1734670849.982827 mmap done 1734670850.103114 msync done 1734670850.103292 munmap done 000502[0] 1734670850.104214 read 0x16114e thru 0x166028 (0x4edb bytes) 1734670850.104317 read done 000503[0] 1734670850.117068 mapread 0x112de2 thru 0x11a809 (0x7a28 bytes) 1734670850.117782 mmap done 1734670850.117871 memcpy done 1734670850.118107 munmap done 000508[1] 1734670850.148713 mapwrite 0x1c5b7f thru 0x1d4477 (0xe8f9 bytes) 1734670850.183449 truncate done 1734670850.184069 mmap done 1734670850.233054 msync done 1734670850.233124 munmap done 000510[1] 1734670850.249967 mapread 0x13a441 thru 0x13c353 (0x1f13 bytes) 1734670850.250209 mmap done 1734670850.260397 memcpy done 1734670850.260944 munmap done 000514[0] 1734670850.291214 trunc from 0x1d4478 to 0x16708c (0x6d3eb bytes) 1734670850.320336 truncate done 000516[1] 1734670850.355612 mapwrite 0x00fc8d thru 0x014c83 (0x4ff7 bytes) 1734670850.362112 mmap done 1734670850.442180 msync done 1734670850.442234 munmap done 000519[0] 1734670850.477928 trunc from 0x16708d to 0x0128f6 (0x154796 bytes) 1734670850.498394 truncate done 000520[1] 1734670850.504722 read 0x011666 thru 0x0128f6 (0x1291 bytes) 1734670850.522308 read done 000523[1] 1734670850.549595 mapwrite 0x1d721a thru 0x1d8a23 (0x180a bytes) 1734670850.580678 truncate done 1734670850.580956 mmap done 1734670850.697871 msync done 1734670850.698070 munmap done 000527[0] 1734670850.744245 mapread 0x0fd126 thru 0x106f1b (0x9df6 bytes) 1734670850.751094 mmap done 1734670850.836054 memcpy done 1734670850.836289 munmap done 000528[1] 1734670850.836696 mapwrite 0x10c1c9 thru 0x117a79 (0xb8b1 bytes) 1734670850.846482 mmap done 1734670850.971453 msync done 1734670850.971570 munmap done 000529[0] 1734670850.983199 write 0x1d3550 thru 0x1d5184 (0x1c35 bytes) 1734670851.001152 write done 000532[0] 1734670851.035613 read 0x19d474 thru 0x1a3a70 (0x65fd bytes) 1734670851.057853 read done 000534[0] 1734670851.059212 read 0x0e2a7c thru 0x0e7253 (0x47d8 bytes) 1734670851.059353 read done 000535[1] 1734670851.059619 write 0x0898ab thru 0x08b285 (0x19db bytes) 1734670851.073973 write done 000538[0] 1734670851.132498 mapwrite 0x1d69a2 thru 0x1e37a3 (0xce02 bytes) 1734670851.173375 truncate done 1734670851.173573 mmap done 1734670851.280301 msync done 1734670851.280472 munmap done 000539[1] 1734670851.284227 mapread 0x1026f5 thru 0x109a18 (0x7324 bytes) 1734670851.290561 mmap done 1734670851.336917 memcpy done 1734670851.337131 munmap done 000540[0] 1734670851.341261 mapwrite 0x144168 thru 0x144eb7 (0xd50 bytes) 1734670851.341435 mmap done 1734670851.390051 msync done 1734670851.390317 munmap done 000541[0] 1734670851.390702 read 0x183eba thru 0x18cb90 (0x8cd7 bytes) 1734670851.416712 read done 000543[0] 1734670851.417333 read 0x0e710b thru 0x0e960e (0x2504 bytes) 1734670851.417403 read done 000544[0] 1734670851.452253 trunc from 0x1e37a4 to 0x0a9af7 (0x139cac bytes) 1734670851.467190 truncate done 000546[1] 1734670851.491306 mapwrite 0x05f155 thru 0x069719 (0xa5c5 bytes) 1734670851.498779 mmap done 1734670851.595854 msync done 1734670851.595932 munmap done 000548[1] 1734670851.635273 write 0x0fb455 thru 0x105da0 (0xa94c bytes) 1734670851.646240 write done 000550[1] 1734670851.646841 write 0x1e19a8 thru 0x1e1e5a (0x4b3 bytes) 1734670851.647273 write done 000551[0] 1734670851.647485 mapwrite 0x043463 thru 0x047dc6 (0x4964 bytes) 1734670851.660830 mmap done 1734670851.756308 msync done 1734670851.756731 munmap done 000553[1] 1734670851.783398 trunc from 0x1e1e5b to 0x0ac486 (0x1359d4 bytes) 1734670851.805532 truncate done 000554[1] 1734670851.807914 write 0x02616c thru 0x03377b (0xd610 bytes) 1734670851.817129 write done 000555[0] 1734670851.834269 mapread 0x007283 thru 0x01124c (0x9fca bytes) 1734670851.842539 mmap done 1734670851.899675 memcpy done 1734670851.899880 munmap done 000558[1] 1734670851.920688 mapwrite 0x1e61d3 thru 0x1f522a (0xf058 bytes) 1734670851.945444 truncate done 1734670851.945852 mmap done 1734670852.061293 msync done 1734670852.061950 munmap done 000559[1] 1734670852.069736 read 0x09aaec thru 0x0a86ce (0xdbe3 bytes) 1734670852.116393 read done 000560[0] 1734670852.152915 write 0x01c8e4 thru 0x02c4e3 (0xfc00 bytes) 1734670852.181111 write done 000561[1] 1734670852.205311 trunc from 0x1f522b to 0x1e3657 (0x11bd3 bytes) 1734670852.240730 truncate done 000562[0] 1734670852.258541 read 0x0310ec thru 0x034eb5 (0x3dca bytes) 1734670852.283976 read done 000563[0] 1734670852.291514 write 0x16b30d thru 0x1797dd (0xe4d1 bytes) 1734670852.341834 write done 000565[1] 1734670852.364930 mapread 0x158f5c thru 0x1642db (0xb380 bytes) 1734670852.370163 mmap done 1734670852.434341 memcpy done 1734670852.434991 munmap done 000567[1] 1734670852.439082 mapwrite 0x0057fa thru 0x00bbfd (0x6404 bytes) 1734670852.439331 mmap done 1734670852.519286 msync done 1734670852.519664 munmap done 000571[1] 1734670852.542018 trunc from 0x1e3658 to 0x0feb88 (0xe4acf bytes) 1734670852.569004 truncate done 000572[0] 1734670852.580331 mapwrite 0x1f0dd8 thru 0x1fd510 (0xc739 bytes) 1734670852.600662 truncate done 1734670852.600836 mmap done 1734670852.683902 msync done 1734670852.684089 munmap done 000574[0] 1734670852.722809 write 0x15130f thru 0x156a4d (0x573f bytes) 1734670852.729585 write done 000577[0] 1734670852.734248 mapwrite 0x1b226b thru 0x1bc1cb (0x9f61 bytes) 1734670852.736280 mmap done 1734670852.815382 msync done 1734670852.815615 munmap done 000578[0] 1734670852.835159 mapread 0x14ba2b thru 0x156a42 (0xb018 bytes) 1734670852.835374 mmap done 1734670852.861210 memcpy done 1734670852.861422 munmap done 000580[0] 1734670852.895862 trunc from 0x1fd511 to 0x1b3b0c (0x49a04 bytes) 1734670852.909467 truncate done 000583[1] 1734670852.945324 read 0x12815e thru 0x12e1c2 (0x6065 bytes) 1734670853.003913 read done 000584[0] 1734670853.015169 trunc from 0x1b3b0d to 0x1a6985 (0xd187 bytes) 1734670853.037378 truncate done 000585[0] 1734670853.054271 read 0x061eb4 thru 0x071777 (0xf8c4 bytes) 1734670853.085679 read done 000587[1] 1734670853.116434 mapwrite 0x0b31af thru 0x0b5d1f (0x2b71 bytes) 1734670853.121004 mmap done 1734670853.211462 msync done 1734670853.212826 munmap done 000588[0] 1734670853.244290 mapread 0x192a62 thru 0x19c73a (0x9cd9 bytes) 1734670853.250132 mmap done 1734670853.250242 memcpy done 1734670853.250391 munmap done 000590[1] 1734670853.307316 read 0x0e9fb0 thru 0x0ee914 (0x4965 bytes) 1734670853.307441 read done 000591[1] 1734670853.332232 read 0x090be8 thru 0x09ea63 (0xde7c bytes) 1734670853.332413 read done 000596[0] 1734670853.396185 mapwrite 0x1be6e9 thru 0x1c475f (0x6077 bytes) 1734670853.414581 truncate done 1734670853.414824 mmap done 1734670853.466367 msync done 1734670853.466415 munmap done 000598[0] 1734670853.471161 mapread 0x01df96 thru 0x02485d (0x68c8 bytes) 1734670853.471333 mmap done 1734670853.498730 memcpy done 1734670853.499527 munmap done 000599[0] 1734670853.519240 mapwrite 0x1d7c2c thru 0x1d9cbd (0x2092 bytes) 1734670853.534565 truncate done 1734670853.535423 mmap done 1734670853.611146 msync done 1734670853.611209 munmap done 000600[1] 1734670853.633996 mapwrite 0x0ec99c thru 0x0ecc46 (0x2ab bytes) 1734670853.641393 mmap done 1734670853.705092 msync done 1734670853.705138 munmap done 000604[1] 1734670853.772387 read 0x018ede thru 0x02794a (0xea6d bytes) 1734670853.772535 read done 000606[0] 1734670853.796482 trunc from 0x1d9cbe to 0x08b7f4 (0x14e4c9 bytes) 1734670853.821097 truncate done 000607[0] 1734670853.831184 trunc from 0x08b7f5 to 0x1955b1 (0x109dbd bytes) 1734670853.839533 truncate done 000609[1] 1734670853.847068 trunc from 0x1955b2 to 0x0e9ae9 (0xabac8 bytes) 1734670853.874052 truncate done 000610[1] 1734670853.879334 mapread 0x0d6d6f thru 0x0deb23 (0x7db5 bytes) 1734670853.879531 mmap done 1734670853.904294 memcpy done 1734670853.904805 munmap done 000612[1] 1734670853.940281 read 0x0af799 thru 0x0bbcf5 (0xc55d bytes) 1734670853.940432 read done 000616[1] 1734670853.973355 read 0x067ade thru 0x072260 (0xa783 bytes) 1734670853.974364 read done 000617[1] 1734670853.990533 read 0x0e1c9c thru 0x0e9ae9 (0x7e4e bytes) 1734670853.990722 read done 000619[1] 1734670853.991599 read 0x0b21ea thru 0x0c1128 (0xef3f bytes) 1734670853.991731 read done 000621[0] 1734670854.009516 write 0x07cb9e thru 0x07cbc2 (0x25 bytes) 1734670854.029497 write done 000622[1] 1734670854.045566 trunc from 0x0e9aea to 0x0f08e8 (0x6dff bytes) 1734670854.083052 truncate done 000623[0] 1734670854.088732 mapwrite 0x035d63 thru 0x03c587 (0x6825 bytes) 1734670854.099688 mmap done 1734670854.207672 msync done 1734670854.207940 munmap done 000624[0] 1734670854.220176 trunc from 0x0f08e9 to 0x0f46eb (0x3e03 bytes) 1734670854.240706 truncate done 000627[1] 1734670854.272057 write 0x0ee622 thru 0x0f3e66 (0x5845 bytes) 1734670854.298174 write done 000629[1] 1734670854.316283 trunc from 0x0f46ec to 0x0c998b (0x2ad60 bytes) 1734670854.336540 truncate done 000631[1] 1734670854.339669 mapread 0x0113a1 thru 0x017ee4 (0x6b44 bytes) 1734670854.339895 mmap done 1734670854.339999 memcpy done 1734670854.340437 munmap done 000633[0] 1734670854.357980 mapread 0x067cc0 thru 0x06ccf1 (0x5032 bytes) 1734670854.365379 mmap done 1734670854.402525 memcpy done 1734670854.402621 munmap done 000634[1] 1734670854.405410 mapwrite 0x13bbab thru 0x146767 (0xabbd bytes) 1734670854.428969 truncate done 1734670854.429157 mmap done 1734670854.552838 msync done 1734670854.553031 munmap done 000635[0] 1734670854.582796 mapread 0x0d0181 thru 0x0d3ff3 (0x3e73 bytes) 1734670854.593747 mmap done 1734670854.660738 memcpy done 1734670854.661240 munmap done 000637[0] 1734670854.705836 read 0x12dc7c thru 0x137e3a (0xa1bf bytes) 1734670854.766732 read done 000640[0] 1734670854.774446 mapwrite 0x1c8d6b thru 0x1d224a (0x94e0 bytes) 1734670854.806907 truncate done 1734670854.807093 mmap done 1734670854.889033 msync done 1734670854.889273 munmap done 000641[1] 1734670854.894179 trunc from 0x1d224b to 0x04aad9 (0x187771 bytes) 1734670854.927574 truncate done 000642[1] 1734670854.945933 mapread 0x0414fb thru 0x041690 (0x196 bytes) 1734670854.946137 mmap done 1734670854.949291 memcpy done 1734670854.949476 munmap done 000644[0] 1734670854.966829 mapread 0x02f38d thru 0x03162b (0x229f bytes) 1734670854.974678 mmap done 1734670854.992027 memcpy done 1734670854.992554 munmap done 000645[0] 1734670854.995248 trunc from 0x04aada to 0x0cefac (0x844d3 bytes) 1734670855.028872 truncate done 000646[1] 1734670855.032690 mapread 0x01bc4b thru 0x021d33 (0x60e9 bytes) 1734670855.041054 mmap done 1734670855.070398 memcpy done 1734670855.070916 munmap done 000647[0] 1734670855.074114 mapwrite 0x16a498 thru 0x17a316 (0xfe7f bytes) 1734670855.093204 truncate done 1734670855.093367 mmap done 1734670855.226981 msync done 1734670855.227193 munmap done 000649[0] 1734670855.231478 write 0x05f1b0 thru 0x066e8a (0x7cdb bytes) 1734670855.241495 write done 000651[1] 1734670855.242687 write 0x0715f4 thru 0x075c81 (0x468e bytes) 1734670855.267292 write done 000652[1] 1734670855.281661 trunc from 0x17a317 to 0x0cc22a (0xae0ec bytes) 1734670855.302269 truncate done 000653[0] 1734670855.318461 mapread 0x02fcda thru 0x03633c (0x6663 bytes) 1734670855.324127 mmap done 1734670855.357611 memcpy done 1734670855.357832 munmap done 000660[0] 1734670855.406722 mapwrite 0x0624be thru 0x06b47f (0x8fc2 bytes) 1734670855.411818 mmap done 1734670855.498868 msync done 1734670855.499595 munmap done 000661[0] 1734670855.517015 read 0x04ba22 thru 0x0507dc (0x4dbb bytes) 1734670855.533406 read done 000663[1] 1734670855.566978 mapwrite 0x0d0cfc thru 0x0d5025 (0x432a bytes) 1734670855.588482 truncate done 1734670855.588675 mmap done 1734670855.665309 msync done 1734670855.665700 munmap done 000664[0] 1734670855.681136 write 0x1c8956 thru 0x1d7d8f (0xf43a bytes) 1734670855.690227 write done 000667[1] 1734670855.741536 read 0x1c1a2c thru 0x1c5c78 (0x424d bytes) 1734670855.816450 read done 000668[1] 1734670855.829663 read 0x010798 thru 0x019efd (0x9766 bytes) 1734670855.829806 read done 000670[1] 1734670855.840373 write 0x05375f thru 0x060b19 (0xd3bb bytes) 1734670855.841711 write done 000671[0] 1734670855.842005 mapwrite 0x0eb9ce thru 0x0fb08c (0xf6bf bytes) 1734670855.848944 mmap done 1734670855.956091 msync done 1734670855.956568 munmap done 000672[0] 1734670855.969098 write 0x0fd993 thru 0x106b24 (0x9192 bytes) 1734670855.991412 write done 000674[1] 1734670856.006227 read 0x0c0fd8 thru 0x0cc18a (0xb1b3 bytes) 1734670856.034906 read done 000679[1] 1734670856.088096 write 0x11977f thru 0x11f50a (0x5d8c bytes) 1734670856.121501 write done 000685[1] 1734670856.169501 mapwrite 0x19080e thru 0x198adc (0x82cf bytes) 1734670856.169805 mmap done 1734670856.225297 msync done 1734670856.225466 munmap done 000686[0] 1734670856.225819 mapread 0x0e128e thru 0x0e8c03 (0x7976 bytes) 1734670856.232906 mmap done 1734670856.259905 memcpy done 1734670856.260094 munmap done 000687[0] 1734670856.269095 mapwrite 0x0c4d49 thru 0x0d0897 (0xbb4f bytes) 1734670856.274921 mmap done 1734670856.408559 msync done 1734670856.408960 munmap done 000688[1] 1734670856.430357 mapread 0x1814b4 thru 0x1896b9 (0x8206 bytes) 1734670856.443244 mmap done 1734670856.443339 memcpy done 1734670856.443468 munmap done 000691[0] 1734670856.521748 read 0x039d15 thru 0x043a10 (0x9cfc bytes) 1734670856.562662 read done 000692[0] 1734670856.572761 mapwrite 0x19655a thru 0x197e16 (0x18bd bytes) 1734670856.583878 mmap done 1734670856.670289 msync done 1734670856.672426 munmap done 000693[0] 1734670856.672813 read 0x13a80f thru 0x1429f2 (0x81e4 bytes) 1734670856.672921 read done 000697[1] 1734670856.709339 write 0x15c83d thru 0x15dcb2 (0x1476 bytes) 1734670856.726603 write done 000699[1] 1734670856.745231 read 0x1818f1 thru 0x18d06a (0xb77a bytes) 1734670856.784191 read done 000702[0] 1734670856.815250 trunc from 0x1d7d90 to 0x12cf3e (0xaae51 bytes) 1734670856.841273 truncate done 000704[1] 1734670856.845460 mapwrite 0x1113db thru 0x11ed2b (0xd951 bytes) 1734670856.852593 mmap done 1734670856.964104 msync done 1734670856.964308 munmap done 000705[0] 1734670856.981296 write 0x141d8a thru 0x14a4fe (0x8775 bytes) 1734670856.989122 write done 000707[0] 1734670856.991941 mapread 0x05e6a1 thru 0x06a44a (0xbdaa bytes) 1734670856.992116 mmap done 1734670856.992264 memcpy done 1734670856.992854 munmap done 000709[1] 1734670857.013740 mapwrite 0x1bc3f5 thru 0x1c6776 (0xa382 bytes) 1734670857.053708 truncate done 1734670857.053915 mmap done 1734670857.150539 msync done 1734670857.150712 munmap done 000710[0] 1734670857.187003 trunc from 0x1c6777 to 0x1f5814 (0x2f09e bytes) 1734670857.218245 truncate done 000711[1] 1734670857.222346 write 0x0a2957 thru 0x0aba2b (0x90d5 bytes) 1734670857.240796 write done 000712[0] 1734670857.266011 trunc from 0x1f5815 to 0x1fca7d (0x7269 bytes) 1734670857.291715 truncate done 000717[1] 1734670857.408666 mapwrite 0x1c3cbf thru 0x1ce849 (0xab8b bytes) 1734670857.425321 mmap done 1734670857.498025 msync done 1734670857.498192 munmap done 000718[1] 1734670857.511920 mapwrite 0x1ce25b thru 0x1dd574 (0xf31a bytes) 1734670857.519032 mmap done 1734670857.571371 msync done 1734670857.571570 munmap done 000719[0] 1734670857.582047 trunc from 0x1fca7e to 0x13995a (0xc3123 bytes) 1734670857.613159 truncate done 000720[1] 1734670857.617031 mapread 0x0c82b5 thru 0x0cd696 (0x53e2 bytes) 1734670857.629929 mmap done 1734670857.685710 memcpy done 1734670857.686151 munmap done 000721[0] 1734670857.710323 mapwrite 0x1f332a thru 0x1fe2de (0xafb5 bytes) 1734670857.734556 truncate done 1734670857.734835 mmap done 1734670857.885800 msync done 1734670857.885969 munmap done 000722[1] 1734670857.901595 trunc from 0x1fe2df to 0x0aa97b (0x153963 bytes) 1734670857.932544 truncate done 000723[1] 1734670857.959422 mapread 0x06a15b thru 0x073cb1 (0x9b57 bytes) 1734670857.959881 mmap done 1734670858.004721 memcpy done 1734670858.005449 munmap done 000724[1] 1734670858.019813 mapread 0x00c3ad thru 0x015396 (0x8fea bytes) 1734670858.020055 mmap done 1734670858.054136 memcpy done 1734670858.054291 munmap done 000729[1] 1734670858.072502 read 0x06509c thru 0x074314 (0xf279 bytes) 1734670858.085463 read done 000732[0] 1734670858.106360 write 0x0d15e8 thru 0x0dc4d1 (0xaeea bytes) 1734670858.116510 write done 000733[1] 1734670858.138157 mapwrite 0x1f0908 thru 0x1fd655 (0xcd4e bytes) 1734670858.170061 truncate done 1734670858.170243 mmap done 1734670858.296497 msync done 1734670858.296686 munmap done 000736[0] 1734670858.301246 mapread 0x0a1c1d thru 0x0a3901 (0x1ce5 bytes) 1734670858.312477 mmap done 1734670858.340666 memcpy done 1734670858.340815 munmap done 000737[1] 1734670858.345324 mapwrite 0x0e5e29 thru 0x0f3b5d (0xdd35 bytes) 1734670858.345618 mmap done 1734670858.436055 msync done 1734670858.436142 munmap done 000738[0] 1734670858.459390 mapread 0x0080f0 thru 0x00883f (0x750 bytes) 1734670858.481441 mmap done 1734670858.501686 memcpy done 1734670858.502219 munmap done 000739[0] 1734670858.519912 mapread 0x155936 thru 0x1590fd (0x37c8 bytes) 1734670858.529302 mmap done 1734670858.551322 memcpy done 1734670858.551581 munmap done 000742[0] 1734670858.556177 mapwrite 0x0df726 thru 0x0e4665 (0x4f40 bytes) 1734670858.556782 mmap done 1734670858.623012 msync done 1734670858.623069 munmap done 000743[1] 1734670858.623383 mapread 0x020a90 thru 0x02c425 (0xb996 bytes) 1734670858.629768 mmap done 1734670858.675730 memcpy done 1734670858.676434 munmap done 000745[1] 1734670858.679813 write 0x11f164 thru 0x125fff (0x6e9c bytes) 1734670858.693643 write done 000751[0] 1734670858.716392 read 0x07bc25 thru 0x07f6c3 (0x3a9f bytes) 1734670858.738031 read done 000759[1] 1734670858.784169 trunc from 0x1fd656 to 0x130b04 (0xccb51 bytes) 1734670858.807119 truncate done 000761[0] 1734670858.812604 trunc from 0x130b05 to 0x038daa (0xf7d5a bytes) 1734670858.851020 truncate done 000762[0] 1734670858.856164 mapread 0x008c70 thru 0x00e6bc (0x5a4d bytes) 1734670858.856400 mmap done 1734670858.881327 memcpy done 1734670858.881544 munmap done 000763[1] 1734670858.881960 read 0x012408 thru 0x021e88 (0xfa81 bytes) 1734670858.897247 read done 000767[0] 1734670858.924269 write 0x16fdd8 thru 0x173d64 (0x3f8d bytes) 1734670858.924962 write done 000768[1] 1734670858.937523 mapread 0x14abba thru 0x14c7f5 (0x1c3c bytes) 1734670858.943004 mmap done 1734670858.975763 memcpy done 1734670858.976075 munmap done 000771[0] 1734670858.987540 read 0x125726 thru 0x13084d (0xb128 bytes) 1734670859.003473 read done 000772[1] 1734670859.003975 mapwrite 0x1a4a5b thru 0x1ab7de (0x6d84 bytes) 1734670859.047560 truncate done 1734670859.048047 mmap done 1734670859.148688 msync done 1734670859.148989 munmap done 000774[1] 1734670859.199441 mapread 0x044000 thru 0x04a801 (0x6802 bytes) 1734670859.200096 mmap done 1734670859.258543 memcpy done 1734670859.258770 munmap done 000775[0] 1734670859.267972 mapread 0x047c28 thru 0x04fba0 (0x7f79 bytes) 1734670859.277353 mmap done 1734670859.369911 memcpy done 1734670859.369972 munmap done 000778[0] 1734670859.395540 mapwrite 0x13d9ee thru 0x14661e (0x8c31 bytes) 1734670859.404556 mmap done 1734670859.508032 msync done 1734670859.508252 munmap done 000782[0] 1734670859.530225 trunc from 0x1ab7df to 0x1f6e1c (0x4b63e bytes) 1734670859.552476 truncate done 000786[0] 1734670859.618520 read 0x1bde56 thru 0x1c4a1e (0x6bc9 bytes) 1734670859.662705 read done 000789[1] 1734670859.736039 write 0x0c51db thru 0x0cf7c4 (0xa5ea bytes) 1734670859.752347 write done 000790[1] 1734670859.775421 trunc from 0x1f6e1d to 0x0eb3c1 (0x10ba5b bytes) 1734670859.793878 truncate done 000791[1] 1734670859.796713 trunc from 0x0eb3c2 to 0x03e2b0 (0xad111 bytes) 1734670859.804390 truncate done 000794[1] 1734670859.808075 read 0x033eae thru 0x03e2b0 (0xa403 bytes) 1734670859.818012 read done 000797[1] 1734670859.847056 mapread 0x039bb9 thru 0x03e2b0 (0x46f8 bytes) 1734670859.847516 mmap done 1734670859.847562 memcpy done 1734670859.847816 munmap done 000799[0] 1734670859.879964 trunc from 0x03e2b1 to 0x0968da (0x5862a bytes) 1734670859.918794 truncate done 000801[0] 1734670859.923874 mapwrite 0x037ef3 thru 0x038a17 (0xb25 bytes) 1734670859.925324 mmap done 1734670859.991999 msync done 1734670859.992955 munmap done 000805[1] 1734670860.058389 mapread 0x012f36 thru 0x013ba4 (0xc6f bytes) 1734670860.068898 mmap done 1734670860.110859 memcpy done 1734670860.111080 munmap done 000806[0] 1734670860.113837 write 0x01f374 thru 0x021bb6 (0x2843 bytes) 1734670860.139909 write done 000807[1] 1734670860.140250 mapread 0x077524 thru 0x07b4de (0x3fbb bytes) 1734670860.157496 mmap done 1734670860.218327 memcpy done 1734670860.219481 munmap done 000809[1] 1734670860.239909 write 0x068bca thru 0x06a61e (0x1a55 bytes) 1734670860.278591 write done 000814[0] 1734670860.383134 mapwrite 0x1b1c56 thru 0x1bfdbf (0xe16a bytes) 1734670860.422593 truncate done 1734670860.422811 mmap done 1734670860.576837 msync done 1734670860.577289 munmap done 000815[0] 1734670860.599969 mapwrite 0x0eef41 thru 0x0f2c22 (0x3ce2 bytes) 1734670860.600245 mmap done 1734670860.680010 msync done 1734670860.680686 munmap done 000817[0] 1734670860.698545 read 0x02cb88 thru 0x0398ec (0xcd65 bytes) 1734670860.733575 read done 000819[1] 1734670860.756965 write 0x1e94f7 thru 0x1f41fe (0xad08 bytes) 1734670860.786257 write done 000823[0] 1734670860.890950 trunc from 0x1f41ff to 0x0be421 (0x135ddd bytes) 1734670860.928384 truncate done 000824[1] 1734670860.954592 mapread 0x083b9d thru 0x08b60b (0x7a6f bytes) 1734670860.964126 mmap done 1734670861.014954 memcpy done 1734670861.015738 munmap done 000825[1] 1734670861.027741 mapwrite 0x06de4d thru 0x0791a0 (0xb354 bytes) 1734670861.037507 mmap done 1734670861.154094 msync done 1734670861.154809 munmap done 000827[1] 1734670861.186617 mapread 0x066164 thru 0x075ae3 (0xf980 bytes) 1734670861.192330 mmap done 1734670861.220423 memcpy done 1734670861.220721 munmap done 000829[0] 1734670861.279129 mapwrite 0x014c53 thru 0x01b28c (0x663a bytes) 1734670861.288768 mmap done 1734670861.394009 msync done 1734670861.394248 munmap done 000830[0] 1734670861.394596 mapread 0x00c269 thru 0x00e75b (0x24f3 bytes) 1734670861.394777 mmap done 1734670861.405142 memcpy done 1734670861.405352 munmap done 000831[0] 1734670861.405862 trunc from 0x0be422 to 0x0f9bbf (0x3b79e bytes) 1734670861.417704 truncate done 000834[0] 1734670861.440328 write 0x10780f thru 0x109105 (0x18f7 bytes) 1734670861.440942 write done 000835[0] 1734670861.441204 write 0x062e49 thru 0x06be23 (0x8fdb bytes) 1734670861.447761 write done 000836[1] 1734670861.461206 write 0x03fb8f thru 0x04dc2d (0xe09f bytes) 1734670861.481289 write done 000838[1] 1734670861.501380 read 0x03078b thru 0x033f3c (0x37b2 bytes) 1734670861.519082 read done 000840[0] 1734670861.541431 mapwrite 0x17e195 thru 0x18dfe3 (0xfe4f bytes) 1734670861.584198 truncate done 1734670861.584498 mmap done 1734670861.754346 msync done 1734670861.754599 munmap done 000846[1] 1734670861.812485 mapwrite 0x0ad6bb thru 0x0b14e4 (0x3e2a bytes) 1734670861.819980 mmap done 1734670861.894002 msync done 1734670861.895883 munmap done 000847[0] 1734670861.930090 read 0x0c3743 thru 0x0cfa2c (0xc2ea bytes) 1734670861.981220 read done 000848[1] 1734670861.981569 mapread 0x0d4ead thru 0x0e4784 (0xf8d8 bytes) 1734670861.988897 mmap done 1734670862.053253 memcpy done 1734670862.053586 munmap done 000849[0] 1734670862.053986 mapwrite 0x1098a0 thru 0x10c449 (0x2baa bytes) 1734670862.054156 mmap done 1734670862.109278 msync done 1734670862.109549 munmap done 000850[0] 1734670862.122883 mapread 0x0fe6fc thru 0x106e18 (0x871d bytes) 1734670862.123298 mmap done 1734670862.123378 memcpy done 1734670862.123411 munmap done 000852[0] 1734670862.136013 read 0x16872f thru 0x16c943 (0x4215 bytes) 1734670862.136123 read done 000853[1] 1734670862.146460 mapwrite 0x092ac4 thru 0x094555 (0x1a92 bytes) 1734670862.152951 mmap done 1734670862.221020 msync done 1734670862.222716 munmap done 000854[1] 1734670862.246277 write 0x06aef3 thru 0x07a02d (0xf13b bytes) 1734670862.257081 write done 000855[1] 1734670862.274537 mapwrite 0x051e5a thru 0x05a095 (0x823c bytes) 1734670862.279073 mmap done 1734670862.375576 msync done 1734670862.375683 munmap done 000859[1] 1734670862.439617 trunc from 0x18dfe4 to 0x1a92de (0x1b2fb bytes) 1734670862.466454 truncate done 000863[0] 1734670862.533331 write 0x1082b6 thru 0x10e1fa (0x5f45 bytes) 1734670862.547772 write done 000864[1] 1734670862.566302 mapread 0x0ceb92 thru 0x0d431b (0x578a bytes) 1734670862.569939 mmap done 1734670862.595871 memcpy done 1734670862.596742 munmap done 000865[1] 1734670862.610325 write 0x1eff10 thru 0x1f782f (0x7920 bytes) 1734670862.635853 write done 000869[1] 1734670862.681533 read 0x1ad37b thru 0x1b7d22 (0xa9a8 bytes) 1734670862.734741 read done 000871[0] 1734670862.735400 mapread 0x0c8113 thru 0x0c8dde (0xccc bytes) 1734670862.739740 mmap done 1734670862.767518 memcpy done 1734670862.767937 munmap done 000872[1] 1734670862.771348 read 0x080355 thru 0x0863da (0x6086 bytes) 1734670862.783913 read done 000874[0] 1734670862.793340 mapwrite 0x13c215 thru 0x13c9cf (0x7bb bytes) 1734670862.802976 mmap done 1734670862.927695 msync done 1734670862.928040 munmap done 000875[1] 1734670862.928430 trunc from 0x1f7830 to 0x1defb3 (0x1887c bytes) 1734670862.971862 truncate done 000876[1] 1734670862.989527 read 0x01e164 thru 0x020ff2 (0x2e8f bytes) 1734670863.019236 read done 000877[0] 1734670863.034564 mapread 0x0eaf90 thru 0x0ee290 (0x3301 bytes) 1734670863.040662 mmap done 1734670863.069553 memcpy done 1734670863.069708 munmap done 000879[0] 1734670863.093697 write 0x0b3ab8 thru 0x0bc814 (0x8d5d bytes) 1734670863.103096 write done 000884[0] 1734670863.165049 mapread 0x108908 thru 0x111b32 (0x922b bytes) 1734670863.175695 mmap done 1734670863.235696 memcpy done 1734670863.236358 munmap done 000893[1] 1734670863.280575 read 0x0b7e71 thru 0x0b9c89 (0x1e19 bytes) 1734670863.336275 read done 000896[0] 1734670863.341407 trunc from 0x1defb4 to 0x0a0025 (0x13ef8e bytes) 1734670863.381994 truncate done 000898[1] 1734670863.429293 mapread 0x088f4e thru 0x098437 (0xf4ea bytes) 1734670863.442063 mmap done 1734670863.502785 memcpy done 1734670863.502858 munmap done 000900[1] 1734670863.516474 trunc from 0x0a0026 to 0x0d8cff (0x38cda bytes) 1734670863.545703 truncate done 000905[1] 1734670863.600997 mapread 0x0ae514 thru 0x0b89bc (0xa4a9 bytes) 1734670863.601604 mmap done 1734670863.638141 memcpy done 1734670863.638548 munmap done 000906[0] 1734670863.650963 trunc from 0x0d8d00 to 0x10ea5b (0x35d5c bytes) 1734670863.682723 truncate done 000908[1] 1734670863.701431 mapread 0x107669 thru 0x10a4a2 (0x2e3a bytes) 1734670863.709855 mmap done 1734670863.731666 memcpy done 1734670863.732148 munmap done 000909[0] 1734670863.735790 trunc from 0x10ea5c to 0x1bdbf0 (0xaf195 bytes) 1734670863.756010 truncate done 000910[0] 1734670863.759155 read 0x0d24bc thru 0x0e12ac (0xedf1 bytes) 1734670863.788793 read done 000914[0] 1734670863.816825 read 0x01a4b9 thru 0x01c3f0 (0x1f38 bytes) 1734670863.816947 read done 000915[0] 1734670863.830175 write 0x060f49 thru 0x06142c (0x4e4 bytes) 1734670863.830495 write done 000917[1] 1734670863.845132 trunc from 0x1bdbf1 to 0x109ef6 (0xb3cfa bytes) 1734670863.882828 truncate done 000918[1] 1734670863.902331 trunc from 0x109ef7 to 0x1c633e (0xbc448 bytes) 1734670863.914484 truncate done 000920[1] 1734670863.919210 mapwrite 0x001657 thru 0x00a4a8 (0x8e52 bytes) 1734670863.919617 mmap done 1734670863.994653 msync done 1734670863.995533 munmap done 000923[0] 1734670864.019881 write 0x13edc5 thru 0x147a47 (0x8c83 bytes) 1734670864.037770 write done 000924[0] 1734670864.050052 write 0x0334d9 thru 0x03a257 (0x6d7f bytes) 1734670864.100271 write done 000926[0] 1734670864.100889 mapwrite 0x070089 thru 0x072ea7 (0x2e1f bytes) 1734670864.101099 mmap done 1734670864.139420 msync done 1734670864.139585 munmap done 000928[0] 1734670864.169272 trunc from 0x1c633f to 0x10b4b7 (0xbae87 bytes) 1734670864.180839 truncate done 000929[0] 1734670864.199002 write 0x1f1b2e thru 0x1f6be6 (0x50b9 bytes) 1734670864.199698 write done 000931[0] 1734670864.232111 trunc from 0x1f6be7 to 0x12b48a (0xcb75c bytes) 1734670864.250762 truncate done 000932[0] 1734670864.272350 trunc from 0x12b48b to 0x0e7e8a (0x43600 bytes) 1734670864.286859 truncate done 000934[0] 1734670864.319383 mapread 0x08497b thru 0x08fe7d (0xb503 bytes) 1734670864.319602 mmap done 1734670864.319711 memcpy done 1734670864.319861 munmap done 000936[1] 1734670864.339784 mapwrite 0x06f0c0 thru 0x07d29b (0xe1dc bytes) 1734670864.346590 mmap done 1734670864.511307 msync done 1734670864.511544 munmap done 000944[1] 1734670864.657407 trunc from 0x0e7e8b to 0x0bf1ff (0x28c8b bytes) 1734670864.683052 truncate done 000946[0] 1734670864.686380 read 0x021b15 thru 0x025832 (0x3d1e bytes) 1734670864.727783 read done 000948[1] 1734670864.733369 write 0x0c0cea thru 0x0cc83a (0xbb51 bytes) 1734670864.746860 write done 000950[0] 1734670864.771507 trunc from 0x0cc83b to 0x01e270 (0xae5ca bytes) 1734670864.811331 truncate done 000951[1] 1734670864.832406 mapwrite 0x0eb6e3 thru 0x0f92c2 (0xdbe0 bytes) 1734670864.859344 truncate done 1734670864.859547 mmap done 1734670864.988307 msync done 1734670864.988971 munmap done 000952[0] 1734670864.992340 write 0x16255d thru 0x1628e9 (0x38d bytes) 1734670865.005072 write done 000953[0] 1734670865.021498 read 0x1321f0 thru 0x13d3e7 (0xb1f8 bytes) 1734670865.064180 read done 000955[1] 1734670865.102859 mapwrite 0x186e56 thru 0x18b462 (0x460d bytes) 1734670865.134618 truncate done 1734670865.134826 mmap done 1734670865.244996 msync done 1734670865.245885 munmap done 000956[1] 1734670865.268121 mapwrite 0x1b2d38 thru 0x1b8ccd (0x5f96 bytes) 1734670865.281902 truncate done 1734670865.282095 mmap done 1734670865.360780 msync done 1734670865.360957 munmap done 000958[1] 1734670865.403591 write 0x0f6b14 thru 0x0fb30d (0x47fa bytes) 1734670865.408091 write done 000963[1] 1734670865.462546 mapread 0x0c701d thru 0x0d5afa (0xeade bytes) 1734670865.463108 mmap done 1734670865.554912 memcpy done 1734670865.555111 munmap done 000964[0] 1734670865.555445 read 0x19e449 thru 0x1a3e67 (0x5a1f bytes) 1734670865.598863 read done 000966[1] 1734670865.620766 read 0x11eb81 thru 0x11ee0e (0x28e bytes) 1734670865.649837 read done 000967[0] 1734670865.675687 read 0x1adcd8 thru 0x1b5752 (0x7a7b bytes) 1734670865.706808 read done 000968[1] 1734670865.707140 write 0x15740d thru 0x1660ea (0xecde bytes) 1734670865.721908 write done 000970[1] 1734670865.743067 mapwrite 0x176b16 thru 0x181b2a (0xb015 bytes) 1734670865.743667 mmap done 1734670865.848185 msync done 1734670865.848260 munmap done 000972[1] 1734670865.848841 read 0x084783 thru 0x0902e5 (0xbb63 bytes) 1734670865.848943 read done 000973[1] 1734670865.849174 write 0x0f3c50 thru 0x0f4e35 (0x11e6 bytes) 1734670865.849436 write done 000974[0] 1734670865.867915 write 0x16b063 thru 0x1783ea (0xd388 bytes) 1734670865.883398 write done 000975[0] 1734670865.897111 read 0x0ac756 thru 0x0bbb1a (0xf3c5 bytes) 1734670865.957847 read done 000978[0] 1734670866.044192 trunc from 0x1b8cce to 0x1d1841 (0x18b74 bytes) 1734670866.093352 truncate done 000981[0] 1734670866.097011 mapwrite 0x179749 thru 0x183a9b (0xa353 bytes) 1734670866.098497 mmap done 1734670866.213371 msync done 1734670866.213615 munmap done 000984[0] 1734670866.231146 read 0x1470ca thru 0x14a86f (0x37a6 bytes) 1734670866.231255 read done 000985[1] 1734670866.250036 write 0x024942 thru 0x02679c (0x1e5b bytes) 1734670866.278266 write done 000987[0] 1734670866.310869 mapwrite 0x1366e8 thru 0x13fca5 (0x95be bytes) 1734670866.317978 mmap done 1734670866.369134 msync done 1734670866.369600 munmap done 000988[1] 1734670866.381102 read 0x1cc9b9 thru 0x1d0ae1 (0x4129 bytes) 1734670866.409062 read done 000994[1] 1734670866.437964 mapwrite 0x179d19 thru 0x17b063 (0x134b bytes) 1734670866.438218 mmap done 1734670866.525688 msync done 1734670866.525968 munmap done 000995 1734670866.526827 [1]close/open 1734670866.531591 close done 1734670866.547405 open done 000998[0] 1734670866.591295 write 0x152c97 thru 0x15677e (0x3ae8 bytes) 1734670866.605254 write done 001000[1] 1734670866.627965 mapread 0x191111 thru 0x19c171 (0xb061 bytes) 1734670866.633489 mmap done 1734670866.673454 memcpy done 1734670866.673857 munmap done All operations completed A-OK! PASS fsx (37s) == sanity-dom test sanity: Run sanity with Data-on-MDT files ========================================================== 00:01:09 (1734670869) mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 oleg431-server: /home/green/git/lustre-release/lustre/tests/except/sanity.0f.ex oleg431-server: /home/green/git/lustre-release/lustre/tests/except/sanity.ex - see client > v2_15_63-134-gdacc4b6d38 for LU-15963, go 312 - need mds1 <= 2.14.55-100-g8a84c7f9c7 for LU-14927, skip 0f - need mds1 < v2_14_55-100-g8a84c7f9c7 for LU-14927, skip 0f excepting tests: 42a 42c 42b 118c 118d 407 119i 411b 130b 130c 130d 130e 130f 130g 312 skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51b === sanity: start setup 00:01:25 (1734670885) === Stopping clients: oleg431-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg431-client.virtnet /mnt/lustre2 opts: oleg431-client.virtnet: executing check_config_client /mnt/lustre oleg431-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg431-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b5960800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5960800.idle_timeout=debug disable quota as required oleg431-server: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity: finish setup 00:01:45 (1734670905) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f14861] preparing for tests involving mounts mke2fs 1.46.2.wc5 (26-Mar-2022) debug=all == sanity test 36a: MDS utime check (mknod, utime) ======= 00:01:49 (1734670909) utime: good mknod times 1734670909 <= 1734670910 <= 1734670910 for /mnt/lustre/f36 utime: good utime mtimes 100000, atime 200000 PASS 36a (4s) == sanity test 36b: OST utime check (open, utime) ======== 00:01:53 (1734670913) utime: good utime mtimes 100000, atime 200000 PASS 36b (4s) == sanity test 36c: non-root MDS utime check (mknod, utime) ========================================================== 00:01:57 (1734670917) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [utime] [/mnt/lustre/d36/f36] utime: good mknod times 1734670918 <= 1734670919 <= 1734670919 for /mnt/lustre/d36/f36 utime: good utime mtimes 100000, atime 200000 PASS 36c (5s) == sanity test 36d: non-root OST utime check (open, utime) ========================================================== 00:02:02 (1734670922) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [utime] [/mnt/lustre/d36/f36] utime: good utime mtimes 100000, atime 200000 PASS 36d (5s) == sanity test 36e: utime on non-owned file (should return error) ========================================================== 00:02:07 (1734670927) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [utime] [/mnt/lustre/d36e.sanity/f36e.sanity] utime: utime(/mnt/lustre/d36e.sanity/f36e.sanity) failed: rc 1: Operation not permitted PASS 36e (5s) == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 00:02:12 (1734670932) fail_loc=0x80000214 Fri Dec 20 00:02:14 EST 2024 1734670934 Fri Dec 20 00:02:15 EST 2024 1734670935 PASS 36f (7s) == sanity test 36g: FMD cache expiry =============================================================================== 00:02:19 (1734670939) FMD max age: 30s FMD before: 5 FMD after: 0 PASS 36g (48s) == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 00:03:07 (1734670987) fail_loc=0x80000227 Fri Dec 20 00:03:08 EST 2024 1734670988 Fri Dec 20 00:03:09 EST 2024 1734670989 PASS 36h (5s) == sanity test 36i: change mtime on striped directory ==== 00:03:12 (1734670992) SKIP: sanity test_36i needs >= 2 MDTs SKIP 36i (2s) == sanity test 39a: mtime changed on create ============== 00:03:15 (1734670995) Succeed in opening file "/mnt/lustre/f39a.sanity2"(flags=O_CREAT) PASS 39a (6s) == sanity test 39b: mtime change on open, link, unlink, rename ================================================================ 00:03:21 (1734671001) repeat after cancel_lru_locks PASS 39b (6s) == sanity test 39c: mtime change on rename ===================================================================================== 00:03:27 (1734671007) repeat after cancel_lru_locks PASS 39c (7s) == sanity test 39d: create, utime, stat ======================================================================================== 00:03:35 (1734671015) repeat after cancel_lru_locks PASS 39d (4s) == sanity test 39e: create, stat, utime, stat ================================================================================== 00:03:39 (1734671019) repeat after cancel_lru_locks PASS 39e (5s) == sanity test 39f: create, stat, sleep, utime, stat =========================================================================== 00:03:44 (1734671024) repeat after cancel_lru_locks PASS 39f (7s) == sanity test 39g: write, chmod, stat ========================================================================================= 00:03:51 (1734671031) repeat after cancel_lru_locks PASS 39g (6s) == sanity test 39h: write, utime within one second, stat ======================================================================= 00:03:57 (1734671037) repeat after cancel_lru_locks PASS 39h (6s) == sanity test 39i: write, rename, stat ======================================================================================== 00:04:03 (1734671043) repeat after cancel_lru_locks PASS 39i (5s) == sanity test 39j: write, rename, close, stat ================================================================================= 00:04:08 (1734671048) debug=-1 debug_mb=150 debug=-1 debug_mb=150 fail_loc=0x80000412 multiop /mnt/lustre/f39j.sanity voO_RDWR:w2097152_c TMPPIPE=/tmp/multiop_open_wait_pipe.14861 repeat after cancel_lru_locks fail_loc=0 debug_mb=21 debug_mb=21 PASS 39j (12s) == sanity test 39k: write, utime, close, stat ================================================================================== 00:04:20 (1734671060) multiop /mnt/lustre/f39k.sanity voO_RDWR:w2097152_c TMPPIPE=/tmp/multiop_open_wait_pipe.14861 repeat after cancel_lru_locks PASS 39k (6s) == sanity test 39l: directory atime update ===================================================================================== 00:04:26 (1734671066) PASS 39l (13s) == sanity test 39m: test atime and mtime before 1970 ===== 00:04:39 (1734671079) repeat after cancel_lru_locks PASS 39m (6s) == sanity test 39n: check that O_NOATIME is honored ====== 00:04:45 (1734671085) 1+0 records in 1+0 records out PASS 39n (15s) == sanity test 39o: directory cached attributes updated after create ========================================================== 00:05:01 (1734671101) a b PASS 39o (5s) == sanity test 39p: remote directory cached attributes updated after create ================================================================== 00:05:05 (1734671105) SKIP: sanity test_39p needs >= 2 MDTs SKIP 39p (2s) == sanity test 39r: lazy atime update on OST ============= 00:05:08 (1734671108) SKIP: sanity test_39r ldiskfs only test SKIP 39r (2s) == sanity test 39q: close won't zero out atime =========== 00:05:10 (1734671110) multiop /mnt/lustre/d39q.sanity vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.14861 PASS 39q (4s) == sanity test 39s: relatime is supported ================ 00:05:14 (1734671114) 192.168.204.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg431-client.virtnet /mnt/lustre (opts:) Starting client: oleg431-client.virtnet: -o relatime 192.168.204.131@tcp:/lustre /mnt/lustre 1+0 records in 1+0 records out Stopping client oleg431-client.virtnet /mnt/lustre (opts:) Starting client: oleg431-client.virtnet: -o user_xattr,flock 192.168.204.131@tcp:/lustre /mnt/lustre PASS 39s (12s) == sanity test 39u: stat race ============================ 00:05:26 (1734671126) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0807127 s, 13.0 MB/s fail_loc=0x80001434 PASS 39u (18s) == sanity test 40: failed open(O_TRUNC) doesn't truncate ======================================================================= 00:05:44 (1734671144) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00548793 s, 746 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [openfile] [-f] [O_WRONLY:O_TRUNC] [/mnt/lustre/f40.sanity] Error in opening file "/mnt/lustre/f40.sanity"(flags=O_WRONLY) 13: Permission denied /mnt/lustre/f40.sanity has type file OK /mnt/lustre/f40.sanity has size 4096 OK PASS 40 (3s) == sanity test 41: test small file write + fstat =============================================================================== 00:05:48 (1734671148) First String: abcdefghijklmnopqr Second String: abcdefghiabcdefghijklmnopqr abcdefghiabcdefghijklmnopqr abcdefghiabcdefghijklmnopqr Pass! PASS 41 (5s) == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 00:05:52 (1734671152) debug=+cache vm.dirty_writeback_centisecs = 0 vm.dirty_writeback_centisecs = 0 vm.dirty_ratio = 50 vm.dirty_background_ratio = 25 100+0 records in 100+0 records out 102400 bytes (102 kB) copied, 0.0428168 s, 2.4 MB/s vm.dirty_writeback_centisecs = 500 vm.dirty_background_ratio = 10 vm.dirty_ratio = 20 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 6784 2200320 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 8192 7530496 1% /mnt/lustre pass grant check: client:6815744 server:6815744 PASS 42d (5s) == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 00:05:57 (1734671157) total: 1000 open/close in 9.55 seconds: 104.75 ops/second total: 934 open/close in 9.42 seconds: 99.17 ops/second 1+0 records in 1+0 records out 489684992 bytes (490 MB) copied, 27.0766 s, 18.1 MB/s osc.lustre-OST0000-osc-ffff8800a6c0a800.cur_dirty_bytes=0 osc.lustre-OST0000-osc-ffff8800a6c0a800.cur_grant_bytes=492044288 osc.lustre-OST0000-osc-ffff8800a6c0a800.cur_dirty_bytes=1048576 osc.lustre-OST0000-osc-ffff8800a6c0a800.cur_grant_bytes=489029632 osc.lustre-OST0000-osc-ffff8800a6c0a800.rpc_stats=0 osc.lustre-OST0000-osc-ffff8800a6c0a800.rpc_stats= snapshot_time: 1734671298.578581782 secs.nsecs start_time: 1734671294.670055156 secs.nsecs elapsed_time: 3.908526626 secs.nsecs read RPCs in flight: 0 write RPCs in flight: 0 DIO RPCs in flight: 0 pending write pages: 0 pending read pages: 0 read write pages per rpc rpcs % cum % | rpcs % cum % 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 33 100 100 read write rpcs in flight rpcs % cum % | rpcs % cum % 1: 0 0 0 | 33 100 100 read write offset rpcs % cum % | rpcs % cum % 0: 0 0 0 | 33 100 100 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 7296 2201344 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 71680 3697664 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 75776 7462912 2% /mnt/lustre pass grant check: client:493748224 server:493748224 PASS 42e (172s) == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 00:08:49 (1734671329) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 5844: /mnt/lustre/d43A.sanity/f43A.sanity: Text file busy PASS 43A (5s) == sanity test 43a: open(RDWR) of file being executed should return -ETXTBSY ========================================================== 00:08:54 (1734671334) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4860: 9356 Terminated $DIR/$tdir/sleep 60 (wd: ~) PASS 43a (5s) == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 00:08:59 (1734671339) truncate: cannot truncate '/mnt/lustre/d43b.sanity/sleep' to length 0: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4860: 9994 Terminated $DIR/$tdir/sleep 60 (wd: ~) PASS 43b (5s) == sanity test 43c: md5sum of copy into lustre =========== 00:09:04 (1734671344) bash: OK PASS 43c (4s) == sanity test 46: dirtying a previously written page ========================================================================== 00:09:09 (1734671349) vm.dirty_writeback_centisecs = 0 vm.dirty_writeback_centisecs = 0 vm.dirty_ratio = 50 vm.dirty_background_ratio = 25 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00650243 s, 630 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00137703 s, 3.0 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00244103 s, 1.7 MB/s vm.dirty_writeback_centisecs = 500 vm.dirty_background_ratio = 10 vm.dirty_ratio = 20 PASS 46 (3s) == sanity test 56r: check lfs find -size works =========== 00:09:13 (1734671353) PASS 56r (5s) == sanity test 101e: check read-ahead for small read(1k) for small files(500k) ========================================================== 00:09:19 (1734671359) Creating 100 500K test files Cancel LRU locks on lustre client to flush the client cache Reset readahead stats llite.lustre-ffff8800a6c0a800.max_cached_mb= users: 5 max_cached_mb: 1865 used_mb: 49 unused_mb: 1816 unevict_mb: 0 reclaim_count: 0 max_read_ahead_mb: 256 used_read_ahead_mb: 0 llite.lustre-ffff8800a6c0a800.read_ahead_stats= snapshot_time 1734671393.980616458 secs.nsecs start_time 1734671383.105911288 secs.nsecs elapsed_time 10.874705170 secs.nsecs hits 12300 samples [pages] misses 200 samples [pages] zero_size_window 100 samples [pages] failed_to_fast_read 200 samples [pages] readahead_pages 100 samples [pages] 123 123 12300 PASS 101e (41s) resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 == sanity test 119a: Short directIO read must return actual read amount ========================================================== 00:10:00 (1734671400) directio on /mnt/lustre/f119a.sanity for 1x524288 bytes PASS PASS 119a (4s) == sanity test 131a: test iov's crossing stripe boundary for writev/readv ========================================================== 00:10:05 (1734671405) PASS 131a (4s) == sanity test 131b: test append writev ================== 00:10:09 (1734671409) /mnt/lustre/f131b.sanity has type file OK /mnt/lustre/f131b.sanity has size 3145728 OK /mnt/lustre/f131b.sanity has type file OK /mnt/lustre/f131b.sanity has size 5767168 OK PASS 131b (4s) == sanity test 131c: test read/write on file w/o objects ========================================================== 00:10:13 (1734671413) Write error: Bad file descriptor (rc = -1, len = 1048576) PASS 131c (4s) == sanity test 131d: test short read ===================== 00:10:17 (1734671417) PASS 131d (5s) == sanity test 131e: test read hitting hole ============== 00:10:22 (1734671422) PASS 131e (5s) == sanity test 150a: truncate/append tests =============== 00:10:28 (1734671428) 1+0 records in 1+0 records out 6096 bytes (6.1 kB) copied, 0.000355137 s, 17.2 MB/s Stopping client oleg431-client.virtnet /mnt/lustre (opts:) Starting client: oleg431-client.virtnet: -o user_xattr,flock 192.168.204.131@tcp:/lustre /mnt/lustre Filesystem 1024-blocks Used Available Capacity Mounted on 192.168.204.131@tcp:/lustre 7542784 10240 7528448 1% /mnt/lustre sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 150a (21s) == sanity test 150b: Verify fallocate (prealloc) functionality ========================================================== 00:10:49 (1734671449) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150b need >= 2.13.57 and ldiskfs for fallocate SKIP 150b (1s) == sanity test 150bb: Verify fallocate modes both zero space ========================================================== 00:10:51 (1734671451) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150bb need >= 2.13.57 and ldiskfs for fallocate SKIP 150bb (2s) == sanity test 150c: Verify fallocate Size and Blocks ==== 00:10:53 (1734671453) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150c need >= 2.13.57 and ldiskfs for fallocate SKIP 150c (2s) == sanity test 150d: Verify fallocate Size and Blocks - Non zero start ========================================================== 00:10:55 (1734671455) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150d need >= 2.13.57 and ldiskfs for fallocate SKIP 150d (2s) == sanity test 150f: Verify fallocate punch functionality ========================================================== 00:10:57 (1734671457) SKIP: sanity test_150f LU-14160: punch mode is not implemented on OSD ZFS SKIP 150f (2s) == sanity test 150g: Verify fallocate punch on large range ========================================================== 00:10:59 (1734671459) SKIP: sanity test_150g LU-14160: punch mode is not implemented on OSD ZFS SKIP 150g (1s) == sanity test 155a: Verify small file correctness: read cache:on write_cache:on ========================================================== 00:11:01 (1734671461) oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='1': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='1': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB) copied, 0.00195003 s, 3.1 MB/s oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 PASS 155a (8s) == sanity test 155b: Verify small file correctness: read cache:on write_cache:off ========================================================== 00:11:08 (1734671468) oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='1': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='0': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB) copied, 0.000703923 s, 8.7 MB/s oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 PASS 155b (8s) == sanity test 155c: Verify small file correctness: read cache:off write_cache:on ========================================================== 00:11:16 (1734671476) oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='0': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='1': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB) copied, 0.00110206 s, 5.5 MB/s oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 PASS 155c (8s) == sanity test 155d: Verify small file correctness: read cache:off write_cache:off ========================================================== 00:11:25 (1734671485) oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='0': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 oleg431-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg431-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='0': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB) copied, 0.00105348 s, 5.8 MB/s oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 oleg431-server: error: set_param: setting : Invalid argument pdsh@oleg431-client: oleg431-server: ssh exited with exit code 22 PASS 155d (8s) == sanity test 207a: can refresh layout at glimpse ======= 00:11:33 (1734671493) 7+0 records in 7+0 records out 28672 bytes (29 kB) copied, 0.0155967 s, 1.8 MB/s fail_loc=0x170 PASS 207a (3s) == sanity test 207b: can refresh layout at open ========== 00:11:37 (1734671497) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00618449 s, 662 kB/s fail_loc=0x171 checksum is 620f0b67a91f7f74151bc5be745b7110 /mnt/lustre/f207b.sanity PASS 207b (3s) == sanity test 241a: bio vs dio ========================== 00:11:41 (1734671501) 1+0 records in 1+0 records out 40960 bytes (41 kB) copied, 0.0106017 s, 3.9 MB/s -rw-r--r-- 1 root root 40960 Dec 20 00:11 /mnt/lustre/f241a.sanity ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58c4800.lock_unused_count=1 PASS 241a (60s) == sanity test 241b: dio vs dio ========================== 00:12:42 (1734671562) 1+0 records in 1+0 records out 40960 bytes (41 kB) copied, 0.0342071 s, 1.2 MB/s -rw-r--r-- 1 root root 40960 Dec 20 00:12 /mnt/lustre/f241b.sanity PASS 241b (33s) == sanity test 251a: Handling short read and write correctly ========================================================== 00:13:15 (1734671595) fail_loc=0xa0001407 fail_val=1 fail_loc=0xa0001407 fail_val=1 PASS 251a (4s) == sanity test 251b: short read restore offset correctly ========================================================== 00:13:19 (1734671599) 4+0 records in 4+0 records out 4096 bytes (4.1 kB) copied, 0.0178659 s, 229 kB/s fail_loc=0x1431 fail_val=5 PASS 251b (9s) == sanity test complete, duration 742 sec ================ 00:13:32 (1734671612) === sanity: start cleanup 00:13:33 (1734671613) === Starting client oleg431-client.virtnet: -o user_xattr,flock 192.168.204.131@tcp:/lustre /mnt/lustre2 Started clients oleg431-client.virtnet: 192.168.204.131@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) Stopping clients: oleg431-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg431-client.virtnet /mnt/lustre2 opts: === sanity: finish cleanup 00:13:38 (1734671618) === debug=super ioctl neterror warning dlmtrace error emerg ha rpctrace vfstrace config console lfsck PASS sanity (753s) == sanity-dom test sanityn: Run sanityn with Data-on-MDT files ========================================================== 00:13:42 (1734671622) mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 oleg431-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 00:13:53 (1734671633) === Starting client oleg431-client.virtnet: -o user_xattr,flock 192.168.204.131@tcp:/lustre /mnt/lustre2 Started clients oleg431-client.virtnet: 192.168.204.131@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg431-client.virtnet: executing check_config_client /mnt/lustre oleg431-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg431-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b58c4800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b605e800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58c4800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b605e800.idle_timeout=debug disable quota as required oleg431-server: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 00:14:11 (1734671651) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209038 s, 50.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f5149] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 00:14:12 (1734671652) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (4s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 00:14:16 (1734671656) -rw-r--r-- 1 root root 0 Dec 20 00:14 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 00:14:19 (1734671659) -rw-r--r-- 1 root root 0 Dec 20 00:14 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 00:14:22 (1734671662) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 00:14:25 (1734671665) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 00:14:28 (1734671668) total 609 -rwxrwxrwx 1 root root 0 Dec 20 00:14 f2a -rwxrwxrwx 1 root root 0 Dec 20 00:14 f2b -rw-r--r-- 1 root root 1906754 Dec 20 00:01 ffsx.sanity-dom -rw-r--r-- 1 root root 0 Dec 20 00:00 ffsx.sanity-dom.fsxgood -rw-r--r-- 1 root root 72897 Dec 20 00:01 ffsx.sanity-dom.fsxlog total 609 -rwxrwxrwx 1 root root 0 Dec 20 00:14 f2a -rwxrwxrwx 1 root root 0 Dec 20 00:14 f2b -rw-r--r-- 1 root root 1906754 Dec 20 00:01 ffsx.sanity-dom -rw-r--r-- 1 root root 0 Dec 20 00:00 ffsx.sanity-dom.fsxgood -rw-r--r-- 1 root root 72897 Dec 20 00:01 ffsx.sanity-dom.fsxlog running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00391576 s, 131 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 00:14:31 (1734671671) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 00:14:33 (1734671673) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.240975 s, 8.7 MB/s /mnt/lustre/f2g.sanityn has 4113 blocks /mnt/lustre2/f2g.sanityn has 4113 blocks PASS 2g (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 00:14:37 (1734671677) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 00:14:40 (1734671680) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 00:14:43 (1734671683) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 00:14:46 (1734671686) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 00:14:49 (1734671689) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 00:14:51 (1734671691) PASS 9a (4s) == sanityn test 9b: append to striped sparse file ======== 00:14:55 (1734671695) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025041 s, 41.9 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0674323 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 00:14:58 (1734671698) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00925807 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0107357 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0082346 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00938753 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0113946 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0066565 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00649639 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00751904 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0123318 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00839935 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00734029 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00627962 s, 0.2 kB/s PASS 10a (4s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 00:15:02 (1734671702) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00684769 s, 449 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0213737 s, 192 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000269515 s, 11.4 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 00:15:05 (1734671705) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.5149 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 00:15:07 (1734671707) warning: '-runtime' deprecated, use '-t runtime' instead - link 728 (time 1734671719.00 total 10.00 last 72.79) - link 1761 (time 1734671729.01 total 20.01 last 103.20) - link 2832 (time 1734671739.02 total 30.02 last 107.05) - link 3949 (time 1734671749.02 total 40.02 last 111.61) - link 5090 (time 1734671759.03 total 50.03 last 114.03) - link 5863 (time 1734671769.03 total 60.03 last 77.28) - link 6692 (time 1734671779.04 total 70.04 last 82.81) - link 7393 (time 1734671789.06 total 80.05 last 70.02) - link 8100 (time 1734671799.07 total 90.06 last 70.63) total: 8803 link in 100.01 seconds: 88.02 ops/second using seed 3939686093 running for 100 seconds - stat 1317 (time 1734671721 ; total 11 ; last 11) - stat 9163 (time 1734671732 ; total 22 ; last 11) - stat 10000 (time 1734671733 ; total 23 ; last 1) - stat 16796 (time 1734671744 ; total 34 ; last 11) - stat 20000 (time 1734671748 ; total 38 ; last 4) - stat 26538 (time 1734671759 ; total 49 ; last 11) - stat 29205 (time 1734671770 ; total 60 ; last 11) - stat 30000 (time 1734671773 ; total 63 ; last 3) - stat 32274 (time 1734671784 ; total 74 ; last 11) - stat 34388 (time 1734671795 ; total 85 ; last 11) - stat 36061 (time 1734671806 ; total 96 ; last 11) total: 37315 stats in 100 seconds: 373.149994 stats/second - unlinked 0 (time 1734671760 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile8803) error: No such file or directory total: 8802 unlinks in 104 seconds: 84.634613 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (18858) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (18861) - No such process PASS 12 (159s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 00:17:47 (1734671867) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.5149 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4860: 19622 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 00:17:50 (1734671870) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4860: 20257 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 00:17:52 (1734671872) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 20891 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 00:17:55 (1734671875) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 21528 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 00:17:58 (1734671878) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 22163 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 00:18:01 (1734671881) fail_loc=0x8000030a PASS 17 (6s) == sanityn test 19: test concurrent uncached read races ========================================================================= 00:18:07 (1734671887) oleg431-server: error: get_param: param_path 'osd-*/lustre-MDT*/read_cache_enable': No such file or directory pdsh@oleg431-client: oleg431-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 00:18:09 (1734671889) PASS 20 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 00:18:13 (1734671893) now is 1734671894 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.5149 new atime is 1734671955 PASS 23 (64s) == sanityn test 27: align non-overlapping extent locks from request ============================================================= 00:19:18 (1734671958) 4+0 records in 4+0 records out 16793600 bytes (17 MB) copied, 0.286521 s, 58.6 MB/s dd 1 started dd 2 started 1+0 records in 1+0 records out 15728640 bytes (16 MB) copied, 0.587723 s, 26.8 MB/s 1+0 records in 1+0 records out 8192 bytes (8.2 kB) copied, 0.0212407 s, 386 kB/s dd 3 finished PASS 27 (7s) == sanityn test 39a: file mtime does not change after rename ========================================================== 00:19:25 (1734671965) repeat after cancel_lru_locks PASS 39a (4s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 00:19:29 (1734671969) repeat after cancel_lru_locks PASS 39b (4s) == sanityn test 39c: check truncate mtime update ================================================================================ 00:19:33 (1734671973) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 00:19:37 (1734671977) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 00:19:41 (1734671981) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.0180063 s, 8.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3518: kill: (29190) - No such process PASS 51a (5s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 00:19:46 (1734671986) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0165709 s, 61.8 kB/s PASS 51c (8s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 00:19:54 (1734671994) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134304 s, 78.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 107a: Basic grouplock conflict =========== 00:19:59 (1734671999) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.181765 s, 57.7 MB/s /mnt/lustre/f107a.sanityn lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 2 lcme_id: 1 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 0 lmm_stripe_size: 1048576 lmm_pattern: mdt lmm_layout_gen: 0 lmm_stripe_offset: 0 lcme_id: 2 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x280000400:0x3db:0x0] } multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.5149 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.5149 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 00:20:04 (1734672004) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.241376 s, 43.4 MB/s /mnt/lustre/f107b.sanityn lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 2 lcme_id: 1 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 0 lmm_stripe_size: 1048576 lmm_pattern: mdt lmm_layout_gen: 0 lmm_stripe_offset: 0 lcme_id: 2 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x240000400:0x3dd:0x0] } multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.5149 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (10s) cleanup: ====================================================== == sanityn test complete, duration 392 sec =============== 00:20:15 (1734672015) === sanityn: start cleanup 00:20:15 (1734672015) === Stopping clients: oleg431-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg431-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 00:20:18 (1734672018) === PASS sanityn (399s) == sanity-dom test complete, duration 1255 sec =========== 00:20:21 (1734672021) === sanity-dom: start cleanup 00:20:22 (1734672022) === Stopping clients: oleg431-client.virtnet /mnt/lustre2 (opts:) === sanity-dom: finish cleanup 00:20:24 (1734672024) ===