-----============= acceptance-small: sanityn ============----- Thu May 1 23:17:11 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg452-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 skipping tests SLOW=no: 33a === sanityn: start setup 23:17:21 (1746155841) === Starting client oleg452-client.virtnet: -o user_xattr,flock 192.168.204.152@tcp:/lustre /mnt/lustre2 Started clients oleg452-client.virtnet: 192.168.204.152@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg452-client.virtnet: executing check_config_client /mnt/lustre oleg452-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg452-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012a4ef000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012a502800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a4ef000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a502800.idle_timeout=debug disable quota as required oleg452-server: oleg452-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanityn: finish setup 23:17:37 (1746155857) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156436 s, 67.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7647] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 23:17:38 (1746155858) PASS 0 (3s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 23:17:41 (1746155861) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 23:17:44 (1746155864) -rw-r--r-- 1 root root 0 May 1 23:17 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 23:17:48 (1746155868) -rw-r--r-- 1 root root 0 May 1 23:17 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 23:17:51 (1746155871) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 23:17:55 (1746155875) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 23:17:58 (1746155878) total 0 -rwxrwxrwx 1 root root 0 May 1 23:17 f2a -rwxrwxrwx 1 root root 0 May 1 23:17 f2b total 0 -rwxrwxrwx 1 root root 0 May 1 23:17 f2a -rwxrwxrwx 1 root root 0 May 1 23:17 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00725815 s, 70.5 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 23:18:02 (1746155882) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (3s) == sanityn test 2g: check blocks update on sync write ==== 23:18:05 (1746155885) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0937814 s, 22.4 MB/s /mnt/lustre/f2g.sanityn has 4096 blocks /mnt/lustre2/f2g.sanityn has 4096 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 23:18:07 (1746155887) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 23:18:10 (1746155890) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 23:18:14 (1746155894) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 23:18:17 (1746155897) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 23:18:21 (1746155901) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 23:18:23 (1746155903) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 23:18:27 (1746155907) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 23:18:30 (1746155910) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219147 s, 47.8 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0329365 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 23:18:32 (1746155912) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00447906 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00564764 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00736414 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00864587 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0127836 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00693298 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00657274 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00764633 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00680519 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00884311 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00551204 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00605526 s, 0.2 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 23:18:36 (1746155916) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0101919 s, 301 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0143989 s, 284 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000267591 s, 11.5 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 23:18:38 (1746155918) striped dir -i1 -c2 -H crush /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7647 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (3s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 23:18:42 (1746155922) warning: '-runtime' deprecated, use '-t runtime' instead - link 731 (time 1746155933.14 total 10.01 last 73.06) - link 1460 (time 1746155943.16 total 20.02 last 72.77) - link 2040 (time 1746155953.17 total 30.03 last 57.93) - link 2650 (time 1746155963.17 total 40.04 last 60.98) - link 3292 (time 1746155973.18 total 50.04 last 64.18) - link 3934 (time 1746155983.18 total 60.04 last 64.18) - link 4572 (time 1746155993.18 total 70.05 last 63.77) - link 5260 (time 1746156003.20 total 80.07 last 68.65) - link 5968 (time 1746156013.21 total 90.07 last 70.78) total: 6717 link in 100.00 seconds: 67.17 ops/second using seed 1730556302 running for 100 seconds - stat 10000 (time 1746155929 ; total 5 ; last 5) - stat 20000 (time 1746155934 ; total 10 ; last 5) - stat 30000 (time 1746155938 ; total 14 ; last 4) - stat 40000 (time 1746155944 ; total 20 ; last 6) - stat 50000 (time 1746155950 ; total 26 ; last 6) - stat 60000 (time 1746155955 ; total 31 ; last 5) - stat 70000 (time 1746155961 ; total 37 ; last 6) - stat 80000 (time 1746155966 ; total 42 ; last 5) - stat 90000 (time 1746155972 ; total 48 ; last 6) - stat 96369 (time 1746155983 ; total 59 ; last 11) - stat 99143 (time 1746155994 ; total 70 ; last 11) - stat 100000 (time 1746155996 ; total 72 ; last 2) - stat 103002 (time 1746156007 ; total 83 ; last 11) - stat 106304 (time 1746156018 ; total 94 ; last 11) total: 108283 stats in 100 seconds: 1082.829956 stats/second - unlinked 0 (time 1746155974 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile6717) error: No such file or directory total: 6716 unlinks in 81 seconds: 82.913582 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (23391) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (23394) - No such process PASS 12 (136s) == sanityn test 13: test directory page revocation ======= 23:20:58 (1746156058) striped dir -i1 -c2 -H crush2 /mnt/lustre/d13 f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 23:21:02 (1746156062) striped dir -i0 -c2 -H all_char /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7647 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 24807 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 23:21:06 (1746156066) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 25454 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 23:21:09 (1746156069) striped dir -i0 -c2 -H crush /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 26101 Terminated $DIR1/$tdir/sleep 60 PASS 14b (4s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 23:21:13 (1746156073) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 26750 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 23:21:16 (1746156076) striped dir -i0 -c2 -H crush /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 27397 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 23:21:20 (1746156080) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7207396 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7207396kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed Waiting for MDT destroys to complete SKIP 15 (13s) == sanityn test 16a: 2500 iterations of dual-mount fsx === 23:21:33 (1746156093) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.222371 s, 37.7 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.202339 s, 41.5 MB/s Chance of close/open is 1 in 50 Seed set to 6095 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn skipping zero size read skipping zero length zero range skipping zero size read skipping zero length punch hole truncating to largest ever: 0x2f01dc fallocating to largest ever: 0x86c86d truncating to largest ever: 0x88d766 truncating to largest ever: 0x92a82e fallocating to largest ever: 0x92db52 truncating to largest ever: 0x9fad60 000200[0] 1746156101.213932 mapread 0x0eb270 thru 0x0ee96e (0x36ff bytes) 000300[0] 1746156103.486061 mapwrite 0x88ec96 thru 0x898b6a (0x9ed5 bytes) 000400[0] 1746156106.061521 punch from 0x52eae6 to 0x539dcf (0xb2ea bytes) 000500[1] 1746156108.313765 write 0x94fa8f thru 0x950c0f (0x1181 bytes) 000600[1] 1746156110.611708 read 0x01ec4a thru 0x02c87e (0xdc35 bytes) 000700[1] 1746156112.935131 punch from 0x855d74 to 0x85892a (0x2bb7 bytes) 000800[1] 1746156115.508887 punch from 0x642a23 to 0x642fcf (0x5ad bytes) 000900[1] 1746156119.012872 write 0x8c6dd6 thru 0x8d2f0d (0xc138 bytes) 001000[0] 1746156121.557353 fallocate 0x54f490 thru 0x551759 (0x22ca bytes) 001100[1] 1746156125.309128 mapread 0x822b6d thru 0x83049d (0xd931 bytes) 001400[0] 1746156134.650550 mapread 0x8fd171 thru 0x90b4df (0xe36f bytes) 001500[0] 1746156136.506620 trunc from 0x13fb6f to 0x14ea69 (0xeefb bytes) 001600[0] 1746156139.258648 mapread 0x693c22 thru 0x69af21 (0x7300 bytes) 001800[0] 1746156145.204325 fallocate 0x0c5442 thru 0x0cd93c (0x84fb bytes) 001900[1] 1746156148.235966 fallocate 0x6da3cf thru 0x6db18a (0xdbc bytes) 002000[1] 1746156151.303186 punch from 0x31e14f to 0x320dd5 (0x2c87 bytes) fallocating to largest ever: 0xa00000 002100[0] 1746156154.137119 mapwrite 0x34953f thru 0x34c0e7 (0x2ba9 bytes) 002200[1] 1746156157.429464 punch from 0x5f6a29 to 0x5fa114 (0x36ec bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 6166 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn skipping zero size read fallocating to largest ever: 0x447c3c truncating to largest ever: 0x62c355 fallocating to largest ever: 0x809e4b truncating to largest ever: 0x9bbac1 truncating to largest ever: 0x9cbd8e 000200[1] 1746156171.213133 trunc from 0x7fe6d8 to 0x2db696 (0x523041 bytes) 000300[1] 1746156173.595363 punch from 0x04d0cd to 0x0500a9 (0x2fdd bytes) truncating to largest ever: 0x9f89a2 000500[1] 1746156177.976769 punch from 0x869235 to 0x8757df (0xc5ab bytes) 000600[0] 1746156179.868883 punch from 0x2ac76f to 0x2b635b (0x9bed bytes) 000700[0] 1746156182.262237 fallocate 0x25853a thru 0x25d14e (0x4c15 bytes) 000800[0] 1746156184.709058 trunc from 0x45ca35 to 0x46ac2d (0xe1f9 bytes) truncating to largest ever: 0x9f8e51 001000[0] 1746156189.643954 mapread 0x435000 thru 0x4415f6 (0xc5f7 bytes) 001100[0] 1746156191.732384 punch from 0x08a8b5 to 0x092491 (0x7bdd bytes) 001200[1] 1746156193.855014 fallocate 0x987ea7 thru 0x98b806 (0x3960 bytes) 001300[1] 1746156196.455072 punch from 0x2e5f9f to 0x2f507d (0xf0df bytes) 001500[1] 1746156201.750145 mapread 0x141000 thru 0x150fc8 (0xffc9 bytes) 001700[1] 1746156206.206963 trunc from 0x4cc765 to 0x4d42db (0x7b77 bytes) 001800[0] 1746156208.284434 mapread 0x05f000 thru 0x062d2f (0x3d30 bytes)