== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 22:13:15 (1767323595) Stopping clients: oleg453-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg453-client.virtnet /mnt/lustre2 (opts:-f) oleg453-server: oleg453-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1053: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1054: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg453-server' oleg453-server: oleg453-server.virtnet: executing load_modules_local oleg453-server: Loading modules from /home/green/git/lustre-release/lustre oleg453-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1053: echo: write error: Device or resource busy oleg453-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1054: echo: write error: Device or resource busy oleg453-server: detected 4 online CPUs by sysfs oleg453-server: MODOPTS_LIBCFS= oleg453-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format ost1: lustre-ost1/ost1 Format mds1: lustre-mdt1/mdt1 Format ost2: lustre-ost2/ost2 start mds service on oleg453-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg453-server: oleg453-server.virtnet: executing set_default_debug -1 all pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg453-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg453-server: oleg453-server.virtnet: executing set_default_debug -1 all pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg453-client.virtnet: -o user_xattr,flock 192.168.204.153@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32648 392 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 985 (time 1767323731.93 total 10.00 last 98.50) - open/close 1918 (time 1767323741.94 total 20.01 last 93.21) - open/close 2817 (time 1767323751.94 total 30.01 last 89.87) - open/close 3906 (time 1767323761.95 total 40.02 last 108.83) - open/close 5037 (time 1767323771.95 total 50.02 last 113.06) - open/close 6075 (time 1767323781.96 total 60.02 last 103.79) - open/close 7228 (time 1767323791.96 total 70.03 last 115.27) - open/close 8368 (time 1767323801.97 total 80.04 last 113.87) - open/close 9462 (time 1767323811.98 total 90.04 last 109.34) - open/close 10000 (time 1767323816.77 total 94.84 last 112.23) - open/close 10999 (time 1767323826.78 total 104.85 last 99.78) - open/close 11997 (time 1767323836.79 total 114.86 last 99.72) - open/close 13110 (time 1767323846.80 total 124.87 last 111.18) - open/close 14177 (time 1767323856.81 total 134.87 last 106.64) - open/close 15270 (time 1767323866.81 total 144.88 last 109.26) - open/close 16205 (time 1767323876.81 total 154.88 last 93.49) - open/close 17224 (time 1767323886.82 total 164.89 last 101.79) - open/close 18240 (time 1767323896.83 total 174.90 last 101.52) - open/close 19232 (time 1767323906.84 total 184.90 last 99.13) - open/close 20000 (time 1767323915.04 total 193.11 last 93.61) - open/close 21094 (time 1767323925.05 total 203.12 last 109.31) - open/close 22157 (time 1767323935.05 total 213.12 last 106.24) - open/close 23071 (time 1767323945.06 total 223.12 last 91.38) - open/close 24046 (time 1767323955.06 total 233.13 last 97.47) - open/close 25074 (time 1767323965.07 total 243.14 last 102.69) - open/close 26181 (time 1767323975.08 total 253.15 last 110.60) - open/close 27080 (time 1767323985.08 total 263.15 last 89.88) - open/close 28030 (time 1767323995.09 total 273.16 last 94.93) - open/close 28930 (time 1767324005.10 total 283.17 last 89.91) - open/close 29872 (time 1767324015.10 total 293.17 last 94.16) - open/close 30824 (time 1767324025.11 total 303.18 last 95.11) - open/close 31777 (time 1767324035.13 total 313.20 last 95.15) - open/close 32776 (time 1767324045.14 total 323.21 last 99.78) - open/close 33767 (time 1767324055.15 total 333.21 last 99.04) - open/close 34756 (time 1767324065.15 total 343.22 last 98.86) - open/close 35904 (time 1767324075.15 total 353.22 last 114.78) - open/close 37060 (time 1767324085.15 total 363.22 last 115.56) - open/close 38208 (time 1767324095.16 total 373.23 last 114.76) - open/close 39368 (time 1767324105.16 total 383.23 last 115.93) - open/close 40000 (time 1767324110.10 total 388.17 last 128.09) - open/close 41179 (time 1767324120.10 total 398.17 last 117.85) - open/close 42261 (time 1767324130.11 total 408.18 last 108.09) - open/close 43277 (time 1767324140.12 total 418.19 last 101.50) - open/close 44318 (time 1767324150.13 total 428.19 last 104.05) - open/close 45294 (time 1767324160.13 total 438.20 last 97.53) - open/close 46536 (time 1767324170.14 total 448.21 last 124.08) - open/close 47614 (time 1767324180.14 total 458.21 last 107.80) - open/close 48801 (time 1767324190.15 total 468.21 last 118.68) - open/close 50000 (time 1767324200.04 total 478.11 last 121.17) - open/close 51152 (time 1767324210.05 total 488.12 last 115.12) - open/close 52269 (time 1767324220.05 total 498.12 last 111.66) - open/close 53389 (time 1767324230.05 total 508.12 last 111.96) - open/close 54571 (time 1767324240.07 total 518.14 last 118.04) - open/close 55535 (time 1767324250.07 total 528.14 last 96.39) - open/close 56602 (time 1767324260.07 total 538.14 last 106.66) - open/close 57783 (time 1767324270.08 total 548.14 last 118.05) - open/close 58914 (time 1767324280.08 total 558.15 last 113.09) - open/close 60000 (time 1767324289.75 total 567.81 last 112.33) - open/close 61107 (time 1767324299.75 total 577.82 last 110.62) - open/close 62246 (time 1767324309.76 total 587.82 last 113.87) - open/close 63448 (time 1767324319.76 total 597.83 last 120.13) - open/close 64513 (time 1767324329.77 total 607.84 last 106.42) total: 64768 open/close in 610.27 seconds: 106.13 ops/second - /unlink 1132 (time 1767324345.11 total 10.00 last 113.19) - /unlink 2288 (time 1767324355.11 total 20.00 last 115.60) - /unlink 3451 (time 1767324365.11 total 30.01 last 116.24) - /unlink 4736 (time 1767324375.12 total 40.01 last 128.42) - /unlink 6042 (time 1767324385.12 total 50.01 last 130.58) - /unlink 7293 (time 1767324395.12 total 60.02 last 125.05) - /unlink 8607 (time 1767324405.14 total 70.03 last 131.24) - /unlink 10000 (time 1767324412.03 total 76.93 last 201.97) - /unlink 12328 (time 1767324422.03 total 86.93 last 232.75) - /unlink 15539 (time 1767324432.04 total 96.93 last 321.06) - /unlink 19073 (time 1767324442.05 total 106.95 last 352.83) - /unlink 20000 (time 1767324444.85 total 109.74 last 331.49) - /unlink 22588 (time 1767324454.85 total 119.74 last 258.77) - /unlink 25456 (time 1767324464.85 total 129.74 last 286.79) - /unlink 28936 (time 1767324474.85 total 139.75 last 347.93) - /unlink 30000 (time 1767324478.45 total 143.34 last 295.96) - /unlink 32737 (time 1767324488.45 total 153.34 last 273.64) - /unlink 35247 (time 1767324498.45 total 163.35 last 250.92) - /unlink 37681 (time 1767324508.46 total 173.35 last 243.32) - /unlink 40000 (time 1767324515.94 total 180.83 last 309.96) - /unlink 42896 (time 1767324525.94 total 190.83 last 289.52) - /unlink 45261 (time 1767324535.94 total 200.84 last 236.47) - /unlink 47684 (time 1767324545.94 total 210.84 last 242.29) - /unlink 50000 (time 1767324553.70 total 218.60 last 298.37) - /unlink 53108 (time 1767324563.71 total 228.60 last 310.75) - /unlink 55914 (time 1767324573.71 total 238.60 last 280.55) - /unlink 59516 (time 1767324583.71 total 248.60 last 360.15) - /unlink 62658 (time 1767324593.71 total 258.60 last 314.16) total: 64768 /unlink in 265.55 seconds: 243.90 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1726 (time 1767324632.44 total 10.00 last 172.59) - open/close 3331 (time 1767324642.45 total 20.00 last 160.49) - open/close 4909 (time 1767324652.45 total 30.00 last 157.76) - open/close 6526 (time 1767324662.45 total 40.01 last 161.68) - open/close 8181 (time 1767324672.46 total 50.01 last 165.34) - open/close 10000 (time 1767324681.81 total 59.36 last 194.57) - open/close 11801 (time 1767324691.81 total 69.37 last 180.06) - open/close 13298 (time 1767324701.81 total 79.37 last 149.69) - open/close 15066 (time 1767324711.81 total 89.37 last 176.77) - open/close 16687 (time 1767324721.82 total 99.38 last 161.95) - open/close 18287 (time 1767324731.84 total 109.40 last 159.64) - open/close 20000 (time 1767324740.87 total 118.42 last 189.85) - open/close 21950 (time 1767324750.87 total 128.42 last 194.99) - open/close 24021 (time 1767324760.87 total 138.43 last 207.05) - open/close 25761 (time 1767324770.87 total 148.43 last 174.00) - open/close 27590 (time 1767324780.87 total 158.43 last 182.87) - open/close 29373 (time 1767324790.87 total 168.43 last 178.26) - open/close 30000 (time 1767324793.71 total 171.27 last 220.74) - open/close 32114 (time 1767324803.71 total 181.27 last 211.38) - open/close 33443 (time 1767324813.72 total 191.28 last 132.82) - open/close 34783 (time 1767324823.73 total 201.28 last 133.91) - open/close 36139 (time 1767324833.74 total 211.29 last 135.50) - open/close 37544 (time 1767324843.74 total 221.29 last 140.45) - open/close 39304 (time 1767324853.74 total 231.30 last 175.99) - open/close 40000 (time 1767324858.03 total 235.58 last 162.29) - open/close 41966 (time 1767324868.03 total 245.59 last 196.54) - open/close 43521 (time 1767324878.03 total 255.59 last 155.48) - open/close 45862 (time 1767324888.03 total 265.59 last 234.05) - open/close 47352 (time 1767324898.04 total 275.59 last 148.96) - open/close 48719 (time 1767324908.04 total 285.60 last 136.68) - open/close 50000 (time 1767324916.53 total 294.08 last 150.89) - open/close 51242 (time 1767324926.53 total 304.09 last 124.14) - open/close 52473 (time 1767324936.53 total 314.09 last 123.09) - open/close 53853 (time 1767324946.54 total 324.10 last 137.93) - open/close 55223 (time 1767324956.55 total 334.10 last 136.89) - open/close 56706 (time 1767324966.55 total 344.11 last 148.21) - open/close 58272 (time 1767324976.56 total 354.11 last 156.58) - open/close 59673 (time 1767324986.56 total 364.11 last 140.09) - open/close 61228 (time 1767324996.56 total 374.11 last 155.50) - open/close 62528 (time 1767325006.56 total 384.12 last 129.96) - open/close 63880 (time 1767325016.56 total 394.12 last 135.19) total: 64768 open/close in 401.08 seconds: 161.49 ops/second - /unlink 2672 (time 1767325035.80 total 10.01 last 267.03) - /unlink 5463 (time 1767325045.80 total 20.01 last 279.08) - /unlink 8657 (time 1767325055.80 total 30.01 last 319.38) - /unlink 10000 (time 1767325058.05 total 32.26 last 597.23) - /unlink 13124 (time 1767325068.05 total 42.26 last 312.29) - /unlink 16362 (time 1767325078.05 total 52.26 last 323.78) - /unlink 20000 (time 1767325086.97 total 61.18 last 407.79) - /unlink 24444 (time 1767325096.98 total 71.18 last 444.35) - /unlink 28840 (time 1767325106.98 total 81.18 last 439.54) - /unlink 30000 (time 1767325110.62 total 84.83 last 318.54) - /unlink 33985 (time 1767325120.62 total 94.83 last 398.49) - /unlink 38426 (time 1767325130.63 total 104.84 last 443.66) - /unlink 40000 (time 1767325134.63 total 108.84 last 393.58) - /unlink 45122 (time 1767325144.63 total 118.84 last 512.16) - /unlink 50000 (time 1767325152.58 total 126.78 last 613.79) - /unlink 55529 (time 1767325162.58 total 136.78 last 552.87) - /unlink 59735 (time 1767325172.58 total 146.79 last 420.55) - /unlink 63243 (time 1767325182.58 total 156.79 last 350.78) total: 64768 /unlink in 161.15 seconds: 401.92 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2001 (time 1767325215.14 total 10.00 last 200.00) - open/close 3910 (time 1767325225.14 total 20.01 last 190.87) - open/close 6045 (time 1767325235.14 total 30.01 last 213.45) - open/close 8000 (time 1767325245.14 total 40.01 last 195.46) - open/close 10000 (time 1767325254.69 total 49.56 last 209.41) - open/close 11590 (time 1767325264.70 total 59.56 last 158.97) - open/close 13646 (time 1767325274.70 total 69.57 last 205.51) - open/close 15465 (time 1767325284.71 total 79.57 last 181.80) - open/close 17030 (time 1767325294.71 total 89.58 last 156.44) - open/close 19437 (time 1767325304.71 total 99.58 last 240.65) - open/close 22231 (time 1767325314.72 total 109.59 last 279.24) - open/close 25014 (time 1767325324.72 total 119.59 last 278.27) - open/close 27502 (time 1767325334.72 total 129.59 last 248.80) - open/close 30000 (time 1767325344.42 total 139.29 last 257.48) - open/close 32528 (time 1767325354.42 total 149.29 last 252.75) - open/close 35347 (time 1767325364.43 total 159.29 last 281.82) - open/close 37679 (time 1767325374.43 total 169.30 last 233.12) - open/close 39575 (time 1767325384.43 total 179.30 last 189.55) - open/close 40000 (time 1767325386.85 total 181.72 last 175.77) - open/close 41992 (time 1767325396.86 total 191.73 last 199.05) - open/close 44083 (time 1767325406.86 total 201.73 last 209.04) - open/close 46447 (time 1767325416.86 total 211.73 last 236.35) - open/close 48666 (time 1767325426.87 total 221.73 last 221.83) - open/close 50000 (time 1767325432.81 total 227.68 last 224.47) - open/close 52153 (time 1767325442.82 total 237.68 last 215.15) - open/close 54165 (time 1767325452.82 total 247.69 last 201.15) - open/close 55991 (time 1767325462.83 total 257.69 last 182.43) - open/close 57764 (time 1767325472.83 total 267.70 last 177.21) - open/close 59615 (time 1767325482.83 total 277.70 last 185.08) - open/close 60000 (time 1767325485.05 total 279.91 last 173.96) - open/close 61711 (time 1767325495.05 total 289.92 last 171.06) - open/close 63234 (time 1767325505.06 total 299.92 last 152.21) total: 64768 open/close in 309.53 seconds: 209.25 ops/second - /unlink 3762 (time 1767325526.36 total 10.00 last 376.16) - /unlink 7725 (time 1767325536.36 total 20.00 last 396.29) - /unlink 10000 (time 1767325541.35 total 24.99 last 455.97) - /unlink 14181 (time 1767325551.35 total 34.99 last 418.06) - /unlink 19484 (time 1767325561.35 total 44.99 last 530.23) - /unlink 23550 (time 1767325571.35 total 54.99 last 406.60) - /unlink 27402 (time 1767325581.35 total 64.99 last 385.14) - /unlink 30000 (time 1767325587.11 total 70.76 last 450.99) - /unlink 34342 (time 1767325597.11 total 80.76 last 434.15) - /unlink 38436 (time 1767325607.11 total 90.76 last 409.36) - /unlink 40000 (time 1767325611.75 total 95.39 last 337.41) - /unlink 43832 (time 1767325621.75 total 105.39 last 383.19) - /unlink 48098 (time 1767325631.75 total 115.39 last 426.51) - /unlink 50000 (time 1767325635.68 total 119.33 last 483.45) - /unlink 54399 (time 1767325645.68 total 129.33 last 439.90) - /unlink 59756 (time 1767325655.68 total 139.33 last 535.67) - /unlink 63547 (time 1767325665.72 total 149.36 last 377.79) total: 64768 /unlink in 152.63 seconds: 424.33 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2844 (time 1767325703.20 total 10.00 last 284.27) - open/close 5385 (time 1767325713.21 total 20.01 last 253.99) - open/close 7678 (time 1767325723.21 total 30.01 last 229.20) - open/close 10000 (time 1767325731.62 total 38.43 last 276.04) - open/close 12210 (time 1767325741.62 total 48.43 last 220.94) - open/close 14241 (time 1767325751.63 total 58.43 last 203.02) - open/close 16085 (time 1767325761.63 total 68.43 last 184.37) - open/close 18423 (time 1767325771.63 total 78.43 last 233.80) - open/close 20000 (time 1767325777.03 total 83.83 last 292.17) - open/close 22565 (time 1767325787.03 total 93.83 last 256.49) - open/close 24729 (time 1767325797.03 total 103.84 last 216.27) - open/close 27418 (time 1767325807.04 total 113.84 last 268.72) - open/close 29894 (time 1767325817.04 total 123.85 last 247.57) - open/close 31830 (time 1767325827.05 total 133.85 last 193.51) - open/close 33873 (time 1767325837.05 total 143.86 last 204.17) - open/close 36190 (time 1767325847.05 total 153.86 last 231.64) - open/close 39070 (time 1767325857.07 total 163.87 last 287.70) - open/close 40000 (time 1767325861.84 total 168.65 last 194.66) - open/close 41944 (time 1767325871.85 total 178.65 last 194.31) - open/close 44639 (time 1767325881.85 total 188.65 last 269.43) - open/close 46755 (time 1767325891.85 total 198.66 last 211.52) - open/close 49178 (time 1767325901.85 total 208.66 last 242.29) - open/close 50000 (time 1767325905.12 total 211.92 last 251.76) - open/close 52951 (time 1767325915.12 total 221.92 last 295.08) - open/close 55674 (time 1767325925.12 total 231.93 last 272.20) - open/close 58156 (time 1767325935.13 total 241.93 last 248.09) - open/close 60000 (time 1767325941.51 total 248.31 last 289.15) - open/close 62963 (time 1767325951.51 total 258.31 last 296.25) total: 64768 open/close in 264.50 seconds: 244.87 ops/second - /unlink 4525 (time 1767325969.25 total 10.00 last 452.48) - /unlink 9165 (time 1767325979.25 total 20.00 last 463.98) - /unlink 13681 (time 1767325989.25 total 30.00 last 451.54) - /unlink 18762 (time 1767325999.25 total 40.00 last 508.03) - /unlink 20000 (time 1767326002.44 total 43.19 last 387.97) - /unlink 24934 (time 1767326012.45 total 53.20 last 493.08) - /unlink 29357 (time 1767326022.45 total 63.20 last 442.23) - /unlink 35243 (time 1767326032.45 total 73.20 last 588.57) - /unlink 40000 (time 1767326040.01 total 80.76 last 629.22) - /unlink 45852 (time 1767326050.01 total 90.77 last 584.99) - /unlink 50000 (time 1767326059.55 total 100.30 last 435.08) - /unlink 54723 (time 1767326069.55 total 110.30 last 472.26) - /unlink 60000 (time 1767326078.77 total 119.52 last 572.17) total: 64768 /unlink in 126.69 seconds: 511.23 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3772 (time 1767326114.70 total 10.00 last 377.16) - open/close 7613 (time 1767326124.70 total 20.00 last 384.10) - open/close 10000 (time 1767326130.80 total 26.10 last 391.63) - open/close 13748 (time 1767326140.80 total 36.10 last 374.77) - open/close 17741 (time 1767326150.80 total 46.10 last 399.21) - open/close 20000 (time 1767326156.63 total 51.93 last 387.69) - open/close 23761 (time 1767326166.63 total 61.93 last 376.09) - open/close 27110 (time 1767326176.63 total 71.93 last 334.85) - open/close 30000 (time 1767326184.38 total 79.67 last 373.12) - open/close 33687 (time 1767326194.38 total 89.67 last 368.68) - open/close 37827 (time 1767326204.38 total 99.68 last 413.86) - open/close 40000 (time 1767326210.38 total 105.68 last 361.94) - open/close 43747 (time 1767326220.38 total 115.68 last 374.68) - open/close 47515 (time 1767326230.39 total 125.68 last 376.76) - open/close 50000 (time 1767326237.44 total 132.74 last 352.13) - open/close 54173 (time 1767326247.44 total 142.74 last 417.26) - open/close 58663 (time 1767326257.45 total 152.74 last 448.93) - open/close 60000 (time 1767326260.47 total 155.76 last 442.63) - open/close 64318 (time 1767326270.47 total 165.76 last 431.70) total: 64768 open/close in 166.76 seconds: 388.40 ops/second - /unlink 8351 (time 1767326282.79 total 10.00 last 835.07) - /unlink 15689 (time 1767326292.79 total 20.00 last 733.77) - /unlink 20000 (time 1767326298.81 total 26.02 last 715.66) - /unlink 26463 (time 1767326308.81 total 36.02 last 646.29) - /unlink 30000 (time 1767326313.36 total 40.57 last 777.45) - /unlink 37871 (time 1767326323.36 total 50.58 last 787.02) - /unlink 40000 (time 1767326326.10 total 53.31 last 777.15) - /unlink 47858 (time 1767326336.10 total 63.32 last 785.77) - /unlink 50000 (time 1767326338.92 total 66.13 last 760.84) - /unlink 58019 (time 1767326348.92 total 76.13 last 801.85) - /unlink 60000 (time 1767326351.14 total 78.35 last 893.20) total: 64768 /unlink in 84.50 seconds: 766.48 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg453-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg453-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg453-server stop mds service on oleg453-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg453-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg453-server' oleg453-server: oleg453-server.virtnet: executing unload_modules_local oleg453-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg453-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg453-client: oleg453-server: ssh exited with exit code 2 pdsh@oleg453-client: oleg453-client: ssh exited with exit code 2 pdsh@oleg453-client: oleg453-client: ssh exited with exit code 2 pdsh@oleg453-client: oleg453-server: ssh exited with exit code 2