== conf-sanity test 110: Adding large_dir with 3-level htree ========================================================== 03:33:34 (1761291214) oleg152-server: debugfs 1.47.1-wc2 (08-Nov-2024) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present Stopping clients: oleg152-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg152-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg152-client: oleg152-server: ssh exited with exit code 2 oleg152-server: oleg152-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing load_modules_local oleg152-server: Loading modules from /home/green/git/lustre-release/lustre oleg152-server: detected 4 online CPUs by sysfs oleg152-server: MODOPTS_LIBCFS= oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg152-server: Force libcfs to create 2 CPU partitions oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg152-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg152-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey MDT params: --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity --backfstype=ldiskfs --device-size=200000 --mkfsoptions=\"-b 1024 -i 65536 -O large_dir -b 4096 -E lazy_itable_init\" --reformat /dev/mapper/mds1_flakey /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing load_modules_local oleg152-server: Loading modules from /home/green/git/lustre-release/lustre oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg152-server: detected 4 online CPUs by sysfs oleg152-server: MODOPTS_LIBCFS= oleg152-server: Force libcfs to create 2 CPU partitions Permanent disk data: Target: lustre:MDT0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x65 (MDT MGS first_time update ) Persistent mount opts: user_xattr,errors=remount-ro Parameters: sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity device size = 2500MB formatting backing filesystem ldiskfs on /dev/mapper/mds1_flakey target name lustre:MDT0000 kilobytes 200000 options -b 1024 -i 65536 -b 4096 -I 1024 -q -O large_dir,uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 1024 -L lustre:MDT0000 -b 1024 -i 65536 -b 4096 -I 1024 -q -O large_dir,uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/mapper/mds1_flakey 200000k Writing CONFIGS/mountdata Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=131072 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 mount lustre on /mnt/lustre..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre Starting client oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre Started clients oleg152-client.virtnet: 192.168.201.152@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) mount lustre on /mnt/lustre2..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre2 Starting client oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre2 Started clients oleg152-client.virtnet: 192.168.201.152@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) mkdir: created directory '/mnt/lustre/d110.conf-sanity' creating 60000 in total creating 40000 hard links to /mnt/lustre/d110.conf-sanity/f110.conf-sanity-0 creating 20000 hard links to /mnt/lustre2/d110.conf-sanity/f110.conf-sanity-1 waiting for PIDs 170711 170726 to complete - link 603 (time 1761291325.81 total 10.00 last 60.28) - link 1272 (time 1761291335.82 total 20.01 last 66.86) - link 1963 (time 1761291345.82 total 30.01 last 69.10) - link 2600 (time 1761291355.82 total 40.02 last 63.66) - link 3242 (time 1761291365.83 total 50.02 last 64.15) - link 3886 (time 1761291375.84 total 60.03 last 64.33) - link 4568 (time 1761291385.86 total 70.05 last 68.10) - link 5248 (time 1761291395.86 total 80.05 last 67.98) - link 6084 (time 1761291405.86 total 90.06 last 83.55) - link 6798 (time 1761291415.88 total 100.07 last 71.32) - link 7503 (time 1761291425.89 total 110.09 last 70.37) - link 8178 (time 1761291435.90 total 120.09 last 67.49) - link 8876 (time 1761291445.91 total 130.10 last 69.73) - link 9506 (time 1761291455.93 total 140.13 last 62.83) - link 10000 (time 1761291465.75 total 149.94 last 50.32) - link 10507 (time 1761291475.76 total 159.95 last 50.66) - link 10907 (time 1761291485.77 total 169.96 last 39.97) - link 11289 (time 1761291495.77 total 179.96 last 38.19) - link 11716 (time 1761291505.79 total 189.98 last 42.63) - link 12157 (time 1761291515.79 total 199.98 last 44.10) - link 12619 (time 1761291525.80 total 209.99 last 46.13) - link 13188 (time 1761291535.82 total 220.01 last 56.82) - link 13822 (time 1761291545.82 total 230.02 last 63.36) - link 14469 (time 1761291555.83 total 240.02 last 64.67) - link 15157 (time 1761291565.84 total 250.03 last 68.76) - link 15797 (time 1761291575.84 total 260.03 last 63.99) - link 16433 (time 1761291585.84 total 270.03 last 63.58) - link 17111 (time 1761291595.85 total 280.04 last 67.74) - link 17769 (time 1761291605.85 total 290.05 last 65.76) - link 18488 (time 1761291615.86 total 300.05 last 71.89) - link 19197 (time 1761291625.86 total 310.06 last 70.85) - link 19901 (time 1761291635.88 total 320.07 last 70.28) total: 20000 link in 321.65 seconds: 62.18 ops/second - link 603 (time 1761291326.02 total 10.01 last 60.26) - link 1287 (time 1761291336.03 total 20.02 last 68.31) - link 1986 (time 1761291346.05 total 30.04 last 69.78) - link 2618 (time 1761291356.06 total 40.05 last 63.10) - link 3249 (time 1761291366.07 total 50.06 last 63.05) - link 3894 (time 1761291376.07 total 60.06 last 64.49) - link 4568 (time 1761291386.07 total 70.06 last 67.40) - link 5249 (time 1761291396.09 total 80.08 last 67.99) - link 6077 (time 1761291406.11 total 90.10 last 82.66) - link 6807 (time 1761291416.12 total 100.10 last 72.94) - link 7507 (time 1761291426.13 total 110.12 last 69.92) - link 8180 (time 1761291436.14 total 120.12 last 67.25) - link 8876 (time 1761291446.14 total 130.12 last 69.60) - link 9493 (time 1761291456.15 total 140.13 last 61.64) - link 9998 (time 1761291466.15 total 150.14 last 50.47) - link 10484 (time 1761291476.16 total 160.15 last 48.55) - link 10890 (time 1761291486.16 total 170.15 last 40.58) - link 11280 (time 1761291496.18 total 180.17 last 38.93) - link 11707 (time 1761291506.18 total 190.17 last 42.69) - link 12140 (time 1761291516.20 total 200.19 last 43.24) - link 12606 (time 1761291526.23 total 210.22 last 46.44) - link 13172 (time 1761291536.25 total 220.24 last 56.50) - link 13806 (time 1761291546.25 total 230.24 last 63.37) - link 14465 (time 1761291556.26 total 240.25 last 65.88) - link 15142 (time 1761291566.27 total 250.25 last 67.64) - link 15796 (time 1761291576.27 total 260.26 last 65.37) - link 16439 (time 1761291586.27 total 270.26 last 64.30) - link 17103 (time 1761291596.27 total 280.26 last 66.37) - link 17763 (time 1761291606.28 total 290.27 last 65.98) - link 18488 (time 1761291616.28 total 300.27 last 72.49) - link 19203 (time 1761291626.28 total 310.27 last 71.47) - link 19897 (time 1761291636.29 total 320.28 last 69.35) - link 20361 (time 1761291646.29 total 330.28 last 46.39) - link 20811 (time 1761291656.30 total 340.29 last 44.97) - link 21249 (time 1761291666.31 total 350.29 last 43.77) - link 21743 (time 1761291676.31 total 360.30 last 49.36) - link 22202 (time 1761291686.33 total 370.32 last 45.83) - link 22606 (time 1761291696.35 total 380.34 last 40.29) - link 23000 (time 1761291706.37 total 390.36 last 39.33) - link 23390 (time 1761291716.39 total 400.37 last 38.95) - link 23754 (time 1761291726.40 total 410.39 last 36.34) - link 24116 (time 1761291736.41 total 420.40 last 36.16) - link 24472 (time 1761291746.43 total 430.42 last 35.53) - link 24806 (time 1761291756.46 total 440.44 last 33.33) - link 25175 (time 1761291766.48 total 450.47 last 36.82) - link 25517 (time 1761291776.50 total 460.49 last 34.13) - link 25886 (time 1761291786.50 total 470.49 last 36.89) - link 26240 (time 1761291796.51 total 480.50 last 35.38) - link 26623 (time 1761291806.52 total 490.51 last 38.27) - link 27006 (time 1761291816.52 total 500.51 last 38.27) - link 27389 (time 1761291826.54 total 510.53 last 38.24) - link 27733 (time 1761291836.56 total 520.55 last 34.32) - link 28101 (time 1761291846.59 total 530.58 last 36.71) - link 28471 (time 1761291856.62 total 540.61 last 36.87) - link 28847 (time 1761291866.63 total 550.61 last 37.59) - link 29228 (time 1761291876.64 total 560.63 last 38.06) - link 29602 (time 1761291886.66 total 570.65 last 37.30) - link 30000 (time 1761291896.35 total 580.34 last 41.07) - link 30400 (time 1761291906.37 total 590.36 last 39.92) - link 30833 (time 1761291916.37 total 600.36 last 43.30) - link 31244 (time 1761291926.38 total 610.37 last 41.09) - link 31661 (time 1761291936.38 total 620.37 last 41.69) - link 32072 (time 1761291946.40 total 630.39 last 41.00) - link 32500 (time 1761291956.42 total 640.40 last 42.75) - link 32894 (time 1761291966.43 total 650.42 last 39.33) - link 33312 (time 1761291976.45 total 660.44 last 41.74) - link 33701 (time 1761291986.45 total 670.44 last 38.90) - link 34116 (time 1761291996.46 total 680.44 last 41.47) - link 34528 (time 1761292006.46 total 690.45 last 41.17) - link 34964 (time 1761292016.48 total 700.47 last 43.52) - link 35375 (time 1761292026.50 total 710.49 last 41.01) - link 35762 (time 1761292036.51 total 720.50 last 38.67) - link 36176 (time 1761292046.53 total 730.51 last 41.34) - link 36572 (time 1761292056.54 total 740.53 last 39.54) - link 36951 (time 1761292066.55 total 750.54 last 37.87) - link 37367 (time 1761292076.55 total 760.54 last 41.59) - link 37777 (time 1761292086.58 total 770.57 last 40.89) - link 38192 (time 1761292096.60 total 780.59 last 41.42) - link 38550 (time 1761292106.62 total 790.60 last 35.73) - link 38899 (time 1761292116.63 total 800.62 last 34.85) - link 39261 (time 1761292126.70 total 810.69 last 35.94) - link 39640 (time 1761292136.72 total 820.71 last 37.85) total: 40000 link in 829.80 seconds: 48.20 ops/second Stopping clients: oleg152-client.virtnet /mnt/lustre (opts:) Stopping client oleg152-client.virtnet /mnt/lustre opts: Stopping clients: oleg152-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg152-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server e2fsck -d -v -t -t -f -n /dev/mapper/mds1_flakey -m8 oleg152-server: e2fsck 1.47.1-wc2 (08-Nov-2024) oleg152-server: Use max possible thread num: 1 instead Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 1], used inodes 278/3120 [Thread 0] jumping to group 0 [Thread 0] e2fsck_pass1_run:2311: increase inode 7 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 81 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 82 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 82 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 83 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 114 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 118 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 118 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 119 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 120 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 121 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 122 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 123 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 123 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 124 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 124 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 125 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 125 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 126 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 126 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 127 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 127 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 128 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 128 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 129 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 130 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 130 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 131 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 131 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 132 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 132 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 133 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 133 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 134 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 134 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 135 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 135 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 136 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 136 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 137 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 137 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 138 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 138 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 139 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 139 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 140 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 140 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 141 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 141 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 142 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 143 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 148 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 148 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 149 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 150 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 150 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 151 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 151 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 152 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 152 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 153 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 153 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 154 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 154 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 155 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 155 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 156 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 156 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 157 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 157 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 158 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 158 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 159 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 159 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 160 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 160 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 161 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 161 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 162 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 162 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 163 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 163 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 164 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 164 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 165 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 165 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 166 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 166 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 167 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 168 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 168 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 169 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 169 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 170 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 170 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 171 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 171 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 172 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 172 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 173 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 173 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 174 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 174 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 175 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 175 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 176 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 176 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 177 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 177 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 178 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 178 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 179 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 179 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 180 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 180 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 181 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 181 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 182 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 182 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 183 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 183 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 185 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 186 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 187 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 193 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2655: increase inode 194 badness 0 to 1 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 194 badness 1 to 3 for 10084 [Thread 0] group 0 finished [Thread 0] e2fsck_pass1_run:2311: increase inode 1596 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1596 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1633 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1633 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1634 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1634 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1635 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1635 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1636 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1636 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1637 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1637 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1638 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2311: increase inode 1639 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1639 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1640 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1640 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1641 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1641 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1642 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2311: increase inode 1643 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1643 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1644 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2675: increase inode 1644 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2311: increase inode 1645 badness 0 to 1 for 1005c [Thread 0] e2fsck_pass1_run:2311: increase inode 1646 badness 0 to 1 for 1005c [Thread 0] group 1 finished [Thread 0] Pass 1: Memory used: 368k/440k (128k/241k), time: 0.02/ 0.00/ 0.00 [Thread 0] Pass 1: I/O read: 1MB, write: 0MB, rate: 44.36MB/s [Thread 0] Scanned group range [0, 1], used inodes 278/283 Pass 2: Checking directory structure Pass 2: Memory used: 368k/0k (83k/286k), time: 0.74/ 0.15/ 0.05 Pass 2: I/O read: 25MB, write: 0MB, rate: 33.94MB/s Pass 3: Checking directory connectivity Peak memory: Memory used: 368k/0k (83k/286k), time: 0.79/ 0.16/ 0.06 Pass 3: Memory used: 368k/0k (82k/287k), time: 0.00/ 0.00/ 0.00 Pass 3: I/O read: 0MB, write: 0MB, rate: 0.00MB/s Pass 4: Checking reference counts Pass 4: Memory used: 368k/0k (78k/291k), time: 0.00/ 0.00/ 0.00 Pass 4: I/O read: 1MB, write: 0MB, rate: 6134.97MB/s Pass 5: Checking group summary information Pass 5: Memory used: 508k/0k (78k/431k), time: 0.01/ 0.00/ 0.00 Pass 5: I/O read: 1MB, write: 0MB, rate: 82.84MB/s 278 inodes used (8.91%, out of 3120) 1 non-contiguous file (0.4%) 1 non-contiguous directory (0.4%) # of inodes with ind/dind/tind blocks: 1/1/0 Extent depth histogram: 203 11453 blocks used (22.91%, out of 50000) 0 bad blocks 1 large file 148 regular files 120 directories 0 character device files 0 block device files 0 fifos 60000 links 0 symbolic links (0 fast symbolic links) 0 sockets ------------ 60266 files Memory used: 508k/0k (81k/428k), time: 0.81/ 0.16/ 0.06 I/O read: 25MB, write: 0MB, rate: 30.71MB/s Stopping clients: oleg152-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg152-client.virtnet /mnt/lustre2 (opts:-f) oleg152-server: oleg152-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing load_modules_local oleg152-server: Loading modules from /home/green/git/lustre-release/lustre oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg152-server: detected 4 online CPUs by sysfs oleg152-server: MODOPTS_LIBCFS= oleg152-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg152-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg152-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg152-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg152-server: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg152-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg152-server: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg152-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server stop mds service on oleg152-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server stop mds service on oleg152-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg152-server