-----============= acceptance-small: sanity-wbc ============----- Wed Jul 24 23:25:41 EDT 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 === sanity-wbc: start setup 23:25:45 (1721877945) === Starting client oleg149-client.virtnet: -o user_xattr,flock oleg149-server@tcp:/lustre /mnt/lustre2 Started clients oleg149-client.virtnet: 192.168.201.149@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg149-client.virtnet: executing check_config_client /mnt/lustre oleg149-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg149-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800a9f8e000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6f9f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9f8e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6f9f000.idle_timeout=debug disable quota as required oleg149-server: oleg149-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanity-wbc: finish setup 23:25:53 (1721877953) === running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6828] nobody:x:99: excepting tests: == sanity-wbc test 1: Basic test for WBC with LAZY flush mode ========================================================== 23:25:54 (1721877954) llite.lustre-ffff8800a9f8e000.wbc.conf=enable llite.lustre-ffff8800b6f9f000.wbc.conf=enable llite.lustre-ffff8800a9f8e000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 llite.lustre-ffff8800b6f9f000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 /mnt/lustre/d1.sanity-wbc, state: (0x0000000f) root protected sync complete, metadata: lustre, flush_mode: lazy_drop 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.000191909 s, 42.7 MB/s 2+0 records in 2+0 records out 8194 bytes (8.2 kB) copied, 0.000205152 s, 39.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.00365333 s, 574 MB/s 2+0 records in 2+0 records out 2097158 bytes (2.1 MB) copied, 0.00326067 s, 643 MB/s oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/file1: File not found by ext2_lookup oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/dir1/file2: File not found by ext2_lookup oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/dir1/dir2: File not found by ext2_lookup oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/file3: File not found by ext2_lookup oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/file4: File not found by ext2_lookup oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/file5: File not found by ext2_lookup oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: ROOT/d1.sanity-wbc/file6: File not found by ext2_lookup stat /mnt/lustre2/d1.sanity-wbc File: '/mnt/lustre2/d1.sanity-wbc' Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272502275 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-24 23:25:54.000000000 -0400 Modify: 2024-07-24 23:25:54.000000000 -0400 Change: 2024-07-24 23:25:54.000000000 -0400 Birth: - oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps Inode: 512112 Type: regular Mode: 0666 Flags: 0x0 Inode: 512111 Type: directory Mode: 0777 Flags: 0x0 Inode: 512110 Type: regular Mode: 0666 Flags: 0x0 Inode: 512109 Type: regular Mode: 0666 Flags: 0x0 Inode: 512108 Type: regular Mode: 0666 Flags: 0x0 Inode: 512107 Type: regular Mode: 0666 Flags: 0x0 stat /mnt/lustre2/d1.sanity-wbc/dir1 File: '/mnt/lustre2/d1.sanity-wbc/dir1' Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272502277 Links: 3 Access: (0777/drwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-24 23:25:54.000000000 -0400 Modify: 2024-07-24 23:25:55.000000000 -0400 Change: 2024-07-24 23:25:55.000000000 -0400 Birth: - oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps oleg149-server: debugfs 1.46.2.wc5 (26-Mar-2022) oleg149-server: /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps Inode: 512113 Type: regular Mode: 0666 Flags: 0x0 Inode: 512114 Type: directory Mode: 0777 Flags: 0x0 /mnt/lustre/d1.sanity-wbc/file1 has size 6 OK /mnt/lustre/d1.sanity-wbc/dir1/file2 has size 8192 OK /mnt/lustre/d1.sanity-wbc/file3 has size 0 OK /mnt/lustre/d1.sanity-wbc/file4 has size 8194 OK /mnt/lustre2/d1.sanity-wbc/file1 has size 6 OK /mnt/lustre2/d1.sanity-wbc/dir1/file2 has size 8192 OK /mnt/lustre2/d1.sanity-wbc/file3 has size 0 OK /mnt/lustre2/d1.sanity-wbc/file4 has size 8194 OK /mnt/lustre2/d1.sanity-wbc/file5 has size 2097152 OK /mnt/lustre2/d1.sanity-wbc/file6 has size 2097158 OK llite.lustre-ffff8800a9f8e000.wbc.conf=disable llite.lustre-ffff8800b6f9f000.wbc.conf=disable PASS 1 (3s) == sanity-wbc test 2: Verify remote read works correctly ========================================================== 23:25:57 (1721877957) llite.lustre-ffff8800a9f8e000.wbc.conf=enable llite.lustre-ffff8800b6f9f000.wbc.conf=enable llite.lustre-ffff8800a9f8e000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 llite.lustre-ffff8800b6f9f000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000198638 s, 5.2 MB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000305695 s, 3.3 MB/s File: '/mnt/lustre2/d2.sanity-wbc' Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272502286 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-24 23:25:57.000000000 -0400 Modify: 2024-07-24 23:25:57.000000000 -0400 Change: 2024-07-24 23:25:57.000000000 -0400 Birth: - File: '/mnt/lustre2/d2.sanity-wbc' Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272502288 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-24 23:25:57.000000000 -0400 Modify: 2024-07-24 23:25:57.000000000 -0400 Change: 2024-07-24 23:25:57.000000000 -0400 Birth: - llite.lustre-ffff8800a9f8e000.wbc.conf=disable llite.lustre-ffff8800b6f9f000.wbc.conf=disable PASS 2 (2s) == sanity-wbc test 3: Remote read for WBC cached regular file with holes ========================================================== 23:25:59 (1721877959) llite.lustre-ffff8800a9f8e000.wbc.conf=enable llite.lustre-ffff8800b6f9f000.wbc.conf=enable llite.lustre-ffff8800a9f8e000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 llite.lustre-ffff8800b6f9f000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000251959 s, 4.1 MB/s /mnt/lustre/d3.sanity-wbc/f3.sanity-wbc has size 1049600 OK File: '/mnt/lustre2/d3.sanity-wbc/f3.sanity-wbc' Size: 1049600 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272502293 Links: 1 Access: (0666/-rw-rw-rw-) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-24 23:25:59.000000000 -0400 Modify: 2024-07-24 23:25:59.000000000 -0400 Change: 2024-07-24 23:25:59.000000000 -0400 Birth: - /mnt/lustre2/d3.sanity-wbc/f3.sanity-wbc has size 1049600 OK 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000196804 s, 5.2 MB/s /mnt/lustre/d3.sanity-wbc/f3.sanity-wbc has size 1049600 OK /mnt/lustre2/d3.sanity-wbc/f3.sanity-wbc has size 1049600 OK llite.lustre-ffff8800a9f8e000.wbc.conf=disable llite.lustre-ffff8800b6f9f000.wbc.conf=disable PASS 3 (1s) == sanity-wbc test 4: Verify unlink() works correctly ==== 23:26:00 (1721877960) llite.lustre-ffff8800a9f8e000.wbc.conf=enable llite.lustre-ffff8800b6f9f000.wbc.conf=enable llite.lustre-ffff8800a9f8e000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 llite.lustre-ffff8800b6f9f000.wbc.conf= cache_mode: memfs flush_mode: lazy_drop max_rpcs: 0 stat: cannot stat '/mnt/lustre/d4.sanity-wbc/f4.sanity-wbc': No such file or directory stat: cannot stat '/mnt/lustre/d4.sanity-wbc/dir2/file2': No such file or directory stat: cannot stat '/mnt/lustre/d4.sanity-wbc/dir2': No such file or directory stat: cannot stat '/mnt/lustre/d4.sanity-wbc': No such file or directory stat: cannot stat '/mnt/lustre/d4.sanity-wbc/dir2': No such file or directory stat: cannot stat '/mnt/lustre/d4.sanity-wbc': No such file or directory stat: cannot stat '/mnt/lustre/d4.sanity-wbc/dir2/file2': No such file or directory llite.lustre-ffff8800a9f8e000.wbc.conf=disable llite.lustre-ffff8800b6f9f000.wbc.conf=disable PASS 4 (3s) == sanity-wbc test 5: Hanle -ENOENT lookup failure correctly ========================================================== 23:26:03 (1721877963) stat: cannot stat 'indirect': No such file or directory PASS 5 (2s) cleanup: ====================================================== == sanity-wbc test complete, duration 24 sec ============= 23:26:05 (1721877965) === sanity-wbc: start cleanup 23:26:05 (1721877965) === Stopping clients: oleg149-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg149-client.virtnet /mnt/lustre2 opts: === sanity-wbc: finish cleanup 23:26:06 (1721877966) ===