== conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 10:36:41 (1761316601) oleg318-server: debugfs 1.47.1-wc2 (08-Nov-2024) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present oleg318-server: debugfs 1.47.1-wc2 (08-Nov-2024) Supported features: dir_prealloc imagic_inodes has_journal ext_attr resize_inode dir_index sparse_super2 fast_commit stable_inodes orphan_file filetype needs_recovery journal_dev meta_bg extent 64bit mmp flex_bg ea_inode dirdata metadata_csum_seed large_dir inline_data encrypt casefold sparse_super large_file huge_file uninit_bg dir_nlink extra_isize quota bigalloc metadata_csum read-only project shared_blocks verity orphan_present umount lustre on /mnt/lustre..... stop ost1 service on oleg318-server stop mds service on oleg318-server stop mds service on oleg318-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg318-server' oleg318-server: oleg318-server.virtnet: executing unload_modules_local oleg318-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg318-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. MDT params: --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity --backfstype=ldiskfs --device-size=2400000 --mkfsoptions=\"-O ea_inode,large_dir -E lazy_itable_init\" --reformat /dev/mapper/mds1_flakey /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg318-server' oleg318-server: oleg318-server.virtnet: executing load_modules_local oleg318-server: Loading modules from /home/green/git/lustre-release/lustre oleg318-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg318-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg318-server: detected 4 online CPUs by sysfs oleg318-server: Force libcfs to create 2 CPU partitions oleg318-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg318-server: quota/lquota options: 'hash_lqs_cur_bits=3' Permanent disk data: Target: lustre:MDT0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x65 (MDT MGS first_time update ) Persistent mount opts: user_xattr,errors=remount-ro Parameters: sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity device size = 2500MB formatting backing filesystem ldiskfs on /dev/mapper/mds1_flakey target name lustre:MDT0000 kilobytes 2400000 options -J size=93 -I 1024 -i 2560 -q -O ea_inode,large_dir,dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init="0" -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:MDT0000 -J size=93 -I 1024 -i 2560 -q -O ea_inode,large_dir,dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init="0" -F /dev/mapper/mds1_flakey 2400000k Writing CONFIGS/mountdata Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg318-server: oleg318-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg318-client: oleg318-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 OST params: --mgsnode=oleg318-server@tcp --fsname=lustre --ost --index=0 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=200000 --mkfsoptions=\"-O large_dir -E lazy_itable_init\" --reformat /dev/mapper/ost1_flakey Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: ,errors=remount-ro Parameters: mgsnode=192.168.203.118@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -I 512 -q -O large_dir,extents,uninit_bg,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init="0" -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -I 512 -q -O large_dir,extents,uninit_bg,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init="0" -F /dev/mapper/ost1_flakey 200000k Writing CONFIGS/mountdata Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 oleg318-server: oleg318-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg318-client: oleg318-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 mount lustre on /mnt/lustre..... Starting client: oleg318-client.virtnet: -o user_xattr,flock oleg318-server@tcp:/lustre /mnt/lustre Starting client oleg318-client.virtnet: -o user_xattr,flock oleg318-server@tcp:/lustre /mnt/lustre Started clients oleg318-client.virtnet: 192.168.203.118@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) mount lustre on /mnt/lustre2..... Starting client: oleg318-client.virtnet: -o user_xattr,flock oleg318-server@tcp:/lustre /mnt/lustre2 Starting client oleg318-client.virtnet: -o user_xattr,flock oleg318-server@tcp:/lustre /mnt/lustre2 Started clients oleg318-client.virtnet: 192.168.203.118@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1326500 1696 1206392 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 142216 1252 126964 1% /mnt/lustre[OST:0] filesystem_summary: 142216 1252 126964 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 960000 272 959728 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 50000 268 49732 1% /mnt/lustre[OST:0] filesystem_summary: 50004 272 49732 1% /mnt/lustre creating 60000 hardlinks to oleg318-client.virtnet-0 creating 60000 hardlinks to oleg318-client.virtnet-1 waiting for PIDs 26968 26982 to complete - link 1013 (time 1761316678.23 total 10.01 last 101.22) - link 2069 (time 1761316688.24 total 20.02 last 105.47) - link 2969 (time 1761316698.25 total 30.03 last 89.95) - link 3948 (time 1761316708.25 total 40.03 last 97.87) - link 4887 (time 1761316718.26 total 50.03 last 93.84) - link 5837 (time 1761316728.26 total 60.04 last 94.97) - link 6742 (time 1761316738.26 total 70.04 last 90.48) - link 7509 (time 1761316748.26 total 80.04 last 76.68) - link 8337 (time 1761316758.27 total 90.05 last 82.75) - link 9162 (time 1761316768.27 total 100.05 last 82.48) - link 10000 (time 1761316778.14 total 109.91 last 84.96) - link 10833 (time 1761316788.15 total 119.93 last 83.20) - link 11605 (time 1761316798.15 total 129.93 last 77.17) - link 12417 (time 1761316808.15 total 139.93 last 81.20) - link 13297 (time 1761316818.16 total 149.94 last 87.91) - link 14206 (time 1761316828.17 total 159.94 last 90.87) - link 15086 (time 1761316838.17 total 169.95 last 87.96) - link 15865 (time 1761316848.18 total 179.96 last 77.84) - link 16681 (time 1761316858.19 total 189.96 last 81.55) - link 17486 (time 1761316868.20 total 199.98 last 80.36) - link 18507 (time 1761316878.21 total 209.99 last 102.04) - link 19461 (time 1761316888.21 total 219.99 last 95.37) - link 20000 (time 1761316894.35 total 226.13 last 87.82) - link 20868 (time 1761316904.37 total 236.14 last 86.66) - link 21719 (time 1761316914.38 total 246.15 last 85.01) - link 22447 (time 1761316924.39 total 256.16 last 72.73) - link 23256 (time 1761316934.39 total 266.17 last 80.88) - link 24116 (time 1761316944.39 total 276.17 last 85.97) - link 25021 (time 1761316954.39 total 286.17 last 90.50) - link 25981 (time 1761316964.40 total 296.18 last 95.93) - link 26818 (time 1761316974.41 total 306.18 last 83.66) - link 27586 (time 1761316984.41 total 316.19 last 76.74) - link 28376 (time 1761316994.42 total 326.19 last 78.98) - link 29056 (time 1761317004.42 total 336.20 last 67.94) - link 29692 (time 1761317014.44 total 346.22 last 63.49) - link 30000 (time 1761317019.43 total 351.21 last 61.74) - link 30772 (time 1761317029.44 total 361.21 last 77.17) - link 31605 (time 1761317039.44 total 371.21 last 83.28) - link 32422 (time 1761317049.44 total 381.22 last 81.64) - link 33173 (time 1761317059.47 total 391.24 last 74.94) - link 33981 (time 1761317069.47 total 401.25 last 80.77) - link 34764 (time 1761317079.47 total 411.25 last 78.30) - link 35521 (time 1761317089.48 total 421.25 last 75.66) - link 36329 (time 1761317099.49 total 431.26 last 80.72) - link 37257 (time 1761317109.49 total 441.27 last 92.74) - link 38097 (time 1761317119.50 total 451.28 last 83.91) - link 38960 (time 1761317129.51 total 461.28 last 86.29) - link 39767 (time 1761317139.51 total 471.29 last 80.65) - link 40000 (time 1761317142.00 total 473.78 last 93.61) - link 40887 (time 1761317152.01 total 483.79 last 88.63) - link 41628 (time 1761317162.02 total 493.80 last 73.99) - link 42501 (time 1761317172.03 total 503.81 last 87.26) - link 43395 (time 1761317182.03 total 513.81 last 89.39) - link 44320 (time 1761317192.03 total 523.81 last 92.47) - link 45102 (time 1761317202.04 total 533.82 last 78.12) - link 45926 (time 1761317212.05 total 543.82 last 82.39) - link 46603 (time 1761317222.05 total 553.83 last 67.68) - link 47307 (time 1761317232.06 total 563.83 last 70.34) - link 48220 (time 1761317242.06 total 573.84 last 91.28) - link 49281 (time 1761317252.07 total 583.84 last 106.04) - link 50000 (time 1761317258.82 total 590.59 last 106.49) - link 50962 (time 1761317268.83 total 600.61 last 96.09) - link 51808 (time 1761317278.83 total 610.61 last 84.58) - link 52549 (time 1761317288.83 total 620.61 last 74.10) - link 53414 (time 1761317298.85 total 630.62 last 86.36) - link 54189 (time 1761317308.86 total 640.64 last 77.39) - link 54987 (time 1761317318.87 total 650.64 last 79.77) - link 55758 (time 1761317328.87 total 660.65 last 77.05) - link 56442 (time 1761317338.88 total 670.66 last 68.36) - link 57052 (time 1761317348.89 total 680.66 last 60.94) - link 57636 (time 1761317358.90 total 690.68 last 58.32) - link 58343 (time 1761317368.91 total 700.68 last 70.66) - link 59068 (time 1761317378.91 total 710.69 last 72.49) - link 59839 (time 1761317388.92 total 720.69 last 77.04) total: 60000 link in 723.03 seconds: 82.98 ops/second - link 1025 (time 1761316678.90 total 10.00 last 102.47) - link 2037 (time 1761316688.91 total 20.01 last 101.09) - link 2950 (time 1761316698.91 total 30.02 last 91.27) - link 3922 (time 1761316708.91 total 40.02 last 97.19) - link 4859 (time 1761316718.92 total 50.03 last 93.62) - link 5811 (time 1761316728.94 total 60.04 last 95.05) - link 6690 (time 1761316738.94 total 70.05 last 87.86) - link 7459 (time 1761316748.95 total 80.06 last 76.82) - link 8308 (time 1761316758.96 total 90.06 last 84.87) - link 9124 (time 1761316768.96 total 100.06 last 81.59) - link 9962 (time 1761316778.96 total 110.07 last 83.76) - link 10769 (time 1761316788.97 total 120.08 last 80.62) - link 11537 (time 1761316798.98 total 130.08 last 76.77) - link 12333 (time 1761316808.99 total 140.09 last 79.53) - link 13215 (time 1761316818.99 total 150.09 last 88.19) - link 14116 (time 1761316828.99 total 160.09 last 90.08) - link 15008 (time 1761316839.00 total 170.10 last 89.13) - link 15771 (time 1761316849.00 total 180.11 last 76.25) - link 16580 (time 1761316859.01 total 190.12 last 80.81) - link 17386 (time 1761316869.02 total 200.13 last 80.51) - link 18412 (time 1761316879.03 total 210.14 last 102.51) - link 19355 (time 1761316889.04 total 220.15 last 94.22) - link 20000 (time 1761316896.19 total 227.30 last 90.20) - link 20843 (time 1761316906.20 total 237.30 last 84.25) - link 21679 (time 1761316916.21 total 247.31 last 83.52) - link 22415 (time 1761316926.22 total 257.33 last 73.49) - link 23240 (time 1761316936.23 total 267.34 last 82.40) - link 24112 (time 1761316946.25 total 277.35 last 87.10) - link 25037 (time 1761316956.25 total 287.36 last 92.45) - link 25969 (time 1761316966.25 total 297.36 last 93.19) - link 26785 (time 1761316976.25 total 307.36 last 81.59) - link 27590 (time 1761316986.26 total 317.37 last 80.44) - link 28359 (time 1761316996.27 total 327.37 last 76.85) - link 29029 (time 1761317006.29 total 337.39 last 66.88) - link 29660 (time 1761317016.30 total 347.41 last 63.01) - link 30000 (time 1761317021.36 total 352.46 last 67.24) - link 30767 (time 1761317031.36 total 362.46 last 76.70) - link 31621 (time 1761317041.36 total 372.47 last 85.33) - link 32409 (time 1761317051.37 total 382.48 last 78.72) - link 33162 (time 1761317061.39 total 392.49 last 75.20) - link 33975 (time 1761317071.40 total 402.50 last 81.21) - link 34779 (time 1761317081.40 total 412.51 last 80.36) - link 35525 (time 1761317091.42 total 422.52 last 74.50) - link 36334 (time 1761317101.42 total 432.53 last 80.88) - link 37266 (time 1761317111.42 total 442.53 last 93.15) - link 38119 (time 1761317121.44 total 452.54 last 85.19) - link 38989 (time 1761317131.44 total 462.55 last 86.96) - link 39799 (time 1761317141.44 total 472.55 last 81.00) - link 40000 (time 1761317143.70 total 474.81 last 89.00) - link 40845 (time 1761317153.70 total 484.81 last 84.47) - link 41638 (time 1761317163.70 total 494.81 last 79.30) - link 42526 (time 1761317173.71 total 504.82 last 88.73) - link 43403 (time 1761317183.72 total 514.82 last 87.65) - link 44339 (time 1761317193.73 total 524.83 last 93.52) - link 45114 (time 1761317203.73 total 534.83 last 77.49) - link 45947 (time 1761317213.74 total 544.84 last 83.21) - link 46592 (time 1761317223.74 total 554.85 last 64.46) - link 47327 (time 1761317233.76 total 564.86 last 73.41) - link 48270 (time 1761317243.76 total 574.86 last 94.28) - link 49381 (time 1761317253.76 total 584.87 last 111.09) - link 50000 (time 1761317259.82 total 590.93 last 102.12) - link 50972 (time 1761317269.82 total 600.93 last 97.16) - link 51788 (time 1761317279.85 total 610.95 last 81.44) - link 52543 (time 1761317289.85 total 620.96 last 75.45) - link 53407 (time 1761317299.86 total 630.96 last 86.36) - link 54169 (time 1761317309.86 total 640.97 last 76.14) - link 54953 (time 1761317319.86 total 650.97 last 78.40) - link 55723 (time 1761317329.87 total 660.97 last 76.97) - link 56393 (time 1761317339.89 total 670.99 last 66.89) - link 56999 (time 1761317349.89 total 681.00 last 60.56) - link 57595 (time 1761317359.90 total 691.01 last 59.54) - link 58266 (time 1761317369.91 total 701.01 last 67.07) - link 59012 (time 1761317379.91 total 711.02 last 74.56) - link 59752 (time 1761317389.93 total 721.03 last 73.90) total: 60000 link in 723.96 seconds: 82.88 ops/second estimate 30429s left after 120000 files / 725s umount lustre on /mnt/lustre2..... Stopping client oleg318-client.virtnet /mnt/lustre2 (opts:-f) umount lustre on /mnt/lustre..... Stopping client oleg318-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg318-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg318-server stop mds service on oleg318-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg318-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg318-server' oleg318-server: oleg318-server.virtnet: executing unload_modules_local oleg318-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg318-server: LNET ready to unload oleg318-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. ETA 30429s after 120000 files / 725s is too long e2fsck -d -v -t -t -f -n /dev/mapper/mds1_flakey -m8 oleg318-server: e2fsck 1.47.1-wc2 (08-Nov-2024) oleg318-server: Use max possible thread num: 1 instead Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 29], used inodes 276/960000 [Thread 0] jumping to group 0 [Thread 0] e2fsck_pass1_run:2675: increase inode 116 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 117 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 118 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 152 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 186 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 187 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 188 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 189 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 190 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 191 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 192 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 193 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 194 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 195 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 196 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 197 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 198 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 199 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 200 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 201 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 202 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 203 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 204 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 205 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 206 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 207 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 208 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 209 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 210 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 211 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 212 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 213 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 214 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 215 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 216 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 217 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 218 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 219 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 220 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 221 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 223 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 224 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 225 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 226 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 227 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 228 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 229 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 230 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 231 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 232 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 233 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 234 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 235 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 236 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 237 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 238 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 239 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 240 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 241 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 242 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 243 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 244 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 245 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 246 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 247 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 248 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 249 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 250 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 251 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 252 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 253 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 254 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 255 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 256 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 257 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 258 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 259 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 260 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 261 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 262 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 263 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 264 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 265 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 266 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 267 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 268 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 269 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 270 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 271 badness 0 to 2 for 10084 [Thread 0] e2fsck_pass1_run:2655: increase inode 276 badness 0 to 1 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 276 badness 1 to 3 for 10084 [Thread 0] e2fsck_pass1_run:2655: increase inode 277 badness 0 to 1 for 10084 [Thread 0] e2fsck_pass1_run:2675: increase inode 277 badness 1 to 3 for 10084 [Thread 0] group 0 finished [Thread 0] group 1 finished [Thread 0] group 2 finished [Thread 0] group 3 finished [Thread 0] group 4 finished [Thread 0] group 5 finished [Thread 0] group 6 finished [Thread 0] group 7 finished [Thread 0] group 8 finished [Thread 0] group 9 finished [Thread 0] group 10 finished [Thread 0] group 11 finished [Thread 0] group 12 finished [Thread 0] group 13 finished [Thread 0] group 14 finished [Thread 0] group 15 finished [Thread 0] group 16 finished [Thread 0] group 17 finished [Thread 0] group 18 finished [Thread 0] group 19 finished [Thread 0] group 20 finished [Thread 0] group 21 finished [Thread 0] group 22 finished [Thread 0] group 23 finished [Thread 0] group 24 finished [Thread 0] group 25 finished [Thread 0] group 26 finished [Thread 0] group 27 finished [Thread 0] group 28 finished [Thread 0] group 29 finished [Thread 0] Pass 1: Memory used: 352k/1212k (126k/227k), time: 0.04/ 0.01/ 0.01 [Thread 0] Pass 1: I/O read: 2MB, write: 0MB, rate: 53.65MB/s [Thread 0] Scanned group range [0, 29], used inodes 276/281 Pass 2: Checking directory structure Pass 2: Memory used: 352k/304k (82k/271k), time: 0.80/ 0.28/ 0.08 Pass 2: I/O read: 49MB, write: 0MB, rate: 61.01MB/s Pass 3: Checking directory connectivity Peak memory: Memory used: 352k/304k (82k/271k), time: 0.93/ 0.31/ 0.09 Pass 3: Memory used: 352k/304k (81k/272k), time: 0.00/ 0.00/ 0.00 Pass 3: I/O read: 0MB, write: 0MB, rate: 0.00MB/s Pass 4: Checking reference counts Pass 4: Memory used: 352k/0k (79k/274k), time: 0.05/ 0.02/ 0.00 Pass 4: I/O read: 1MB, write: 0MB, rate: 18.60MB/s Pass 5: Checking group summary information Pass 5: Memory used: 492k/0k (78k/415k), time: 0.01/ 0.00/ 0.00 Pass 5: I/O read: 1MB, write: 0MB, rate: 83.18MB/s 276 inodes used (0.03%, out of 960000) 3 non-contiguous files (1.1%) 1 non-contiguous directory (0.4%) # of inodes with ind/dind/tind blocks: 1/1/0 278401 blocks used (46.40%, out of 600000) 0 bad blocks 1 large file 148 regular files 118 directories 0 character device files 0 block device files 0 fifos 120000 links 0 symbolic links (0 fast symbolic links) 0 sockets ------------ 120264 files Memory used: 492k/0k (80k/413k), time: 1.02/ 0.33/ 0.09 I/O read: 49MB, write: 0MB, rate: 47.93MB/s pdsh@oleg318-client: oleg318-client: ssh exited with exit code 2 pdsh@oleg318-client: oleg318-server: ssh exited with exit code 2 pdsh@oleg318-client: oleg318-server: ssh exited with exit code 2 pdsh@oleg318-client: oleg318-client: ssh exited with exit code 2 Stopping clients: oleg318-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg318-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg318-client: oleg318-server: ssh exited with exit code 2 oleg318-server: oleg318-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg318-server' oleg318-server: oleg318-server.virtnet: executing load_modules_local oleg318-server: Loading modules from /home/green/git/lustre-release/lustre oleg318-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg318-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg318-server: detected 4 online CPUs by sysfs oleg318-server: Force libcfs to create 2 CPU partitions oleg318-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg318-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey