== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 01:48:47 (1770446927) Stopping clients: oleg236-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg236-client.virtnet /mnt/lustre2 (opts:-f) oleg236-server: oleg236-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: MODOPTS_LIBCFS= oleg236-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg236-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg236-server: oleg236-server.virtnet: executing set_default_debug -1 all pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg236-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg236-server: oleg236-server.virtnet: executing set_default_debug -1 all pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg236-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg236-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg236-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg236-server: oleg236-server.virtnet: executing set_default_debug -1 all pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg236-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg236-client.virtnet: -o user_xattr,flock 192.168.202.136@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65969 529 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 2578 (time 1770446980.02 total 10.00 last 257.76) - open/close 5186 (time 1770446990.03 total 20.00 last 260.75) - open/close 7872 (time 1770447000.03 total 30.01 last 268.55) - open/close 10000 (time 1770447005.53 total 35.51 last 386.85) - open/close 14872 (time 1770447015.53 total 45.51 last 487.14) - open/close 19542 (time 1770447025.53 total 55.51 last 466.92) - open/close 24469 (time 1770447035.53 total 65.51 last 492.61) - open/close 29300 (time 1770447045.53 total 75.51 last 483.05) - open/close 34220 (time 1770447055.54 total 85.51 last 491.93) - open/close 38992 (time 1770447065.54 total 95.51 last 477.14) - open/close 43995 (time 1770447075.54 total 105.52 last 500.22) - open/close 48911 (time 1770447085.54 total 115.52 last 491.60) - open/close 50000 (time 1770447087.90 total 117.87 last 461.65) - open/close 54757 (time 1770447097.90 total 127.88 last 475.65) - open/close 58270 (time 1770447107.90 total 137.88 last 351.30) - open/close 60000 (time 1770447111.53 total 141.51 last 476.59) - open/close 64715 (time 1770447121.53 total 151.51 last 471.43) total: 64768 open/close in 151.62 seconds: 427.18 ops/second - /unlink 7917 (time 1770447133.08 total 10.00 last 791.68) - /unlink 10000 (time 1770447135.67 total 12.60 last 802.38) - /unlink 17803 (time 1770447145.67 total 22.60 last 780.23) - /unlink 20000 (time 1770447148.42 total 25.35 last 799.24) - /unlink 27737 (time 1770447158.42 total 35.35 last 773.63) - /unlink 30000 (time 1770447161.43 total 38.36 last 752.06) - /unlink 35291 (time 1770447171.43 total 48.36 last 529.07) - /unlink 40000 (time 1770447179.61 total 56.54 last 575.78) - /unlink 45961 (time 1770447189.61 total 66.54 last 596.06) - /unlink 50000 (time 1770447195.47 total 72.39 last 689.88) - /unlink 53681 (time 1770447205.47 total 82.39 last 368.00) - /unlink 57297 (time 1770447215.47 total 92.39 last 361.58) - /unlink 60000 (time 1770447222.98 total 99.90 last 359.92) - /unlink 63089 (time 1770447232.98 total 109.90 last 308.88) total: 64768 /unlink in 115.40 seconds: 561.24 ops/second Waiting for MDT destroys to complete oleg236-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 - open/close 1014 (time 1770447260.26 total 10.01 last 101.30) - open/close 2227 (time 1770447270.26 total 20.01 last 121.29) - open/close 3599 (time 1770447280.26 total 30.02 last 137.14) - open/close 5090 (time 1770447290.27 total 40.02 last 149.06) - open/close 6433 (time 1770447300.27 total 50.03 last 134.20) - open/close 7597 (time 1770447310.28 total 60.03 last 116.32) - open/close 8689 (time 1770447320.29 total 70.04 last 109.12) - open/close 10000 (time 1770447329.20 total 78.95 last 147.18) - open/close 11439 (time 1770447339.20 total 88.95 last 143.82) - open/close 13091 (time 1770447349.20 total 98.96 last 165.16) - open/close 16100 (time 1770447359.21 total 108.96 last 300.80) - open/close 17429 (time 1770447369.21 total 118.96 last 132.82) - open/close 18996 (time 1770447379.22 total 128.97 last 156.63) - open/close 20000 (time 1770447385.99 total 135.74 last 148.19) - open/close 21203 (time 1770447396.00 total 145.75 last 120.19) - open/close 22240 (time 1770447406.01 total 155.76 last 103.67) - open/close 23229 (time 1770447416.01 total 165.76 last 98.87) - open/close 24238 (time 1770447426.01 total 175.76 last 100.88) - open/close 25479 (time 1770447436.01 total 185.76 last 124.10) - open/close 26605 (time 1770447446.03 total 195.78 last 112.41) - open/close 27765 (time 1770447456.03 total 205.78 last 115.95) - open/close 28985 (time 1770447466.03 total 215.78 last 121.99) - open/close 30000 (time 1770447473.98 total 223.73 last 127.69) - open/close 31291 (time 1770447483.99 total 233.74 last 129.02) - open/close 32951 (time 1770447493.99 total 243.74 last 165.98) - open/close 34767 (time 1770447504.00 total 253.75 last 181.44) - open/close 36324 (time 1770447514.00 total 263.75 last 155.63) - open/close 37741 (time 1770447524.00 total 273.75 last 141.69) - open/close 39709 (time 1770447534.00 total 283.75 last 196.79) - open/close 41611 (time 1770447544.01 total 293.76 last 190.18) - open/close 43564 (time 1770447554.01 total 303.76 last 195.20) - open/close 44913 (time 1770447564.02 total 313.77 last 134.77) - open/close 46253 (time 1770447574.02 total 323.77 last 134.00) - open/close 47508 (time 1770447584.02 total 333.77 last 125.45) - open/close 48963 (time 1770447594.03 total 343.79 last 145.35) - open/close 50000 (time 1770447602.76 total 352.51 last 118.85) - open/close 51132 (time 1770447612.77 total 362.52 last 113.14) - open/close 52265 (time 1770447622.77 total 372.52 last 113.27) - open/close 53414 (time 1770447632.78 total 382.53 last 114.81) - open/close 54707 (time 1770447642.78 total 392.53 last 129.26) - open/close 55837 (time 1770447652.78 total 402.53 last 112.96) - open/close 57008 (time 1770447662.78 total 412.53 last 117.10) - open/close 58512 (time 1770447672.79 total 422.54 last 150.27) - open/close 59770 (time 1770447682.79 total 432.54 last 125.79) - open/close 60883 (time 1770447692.80 total 442.55 last 111.25) - open/close 62079 (time 1770447702.80 total 452.55 last 119.60) - open/close 63472 (time 1770447712.80 total 462.55 last 139.27) total: 64768 open/close in 471.65 seconds: 137.32 ops/second - /unlink 2826 (time 1770447733.94 total 10.01 last 282.46) - /unlink 5405 (time 1770447743.94 total 20.01 last 257.86) - /unlink 8133 (time 1770447753.94 total 30.01 last 272.65) - /unlink 10000 (time 1770447760.88 total 36.95 last 269.11) - /unlink 13061 (time 1770447770.88 total 46.95 last 306.03) - /unlink 15923 (time 1770447780.89 total 56.96 last 286.10) - /unlink 18619 (time 1770447790.89 total 66.96 last 269.58) - /unlink 20000 (time 1770447796.46 total 72.53 last 247.93) - /unlink 22598 (time 1770447806.46 total 82.53 last 259.75) - /unlink 25139 (time 1770447816.46 total 92.53 last 254.08) - /unlink 27880 (time 1770447826.47 total 102.53 last 273.97) - /unlink 30000 (time 1770447833.77 total 109.84 last 290.17) - /unlink 32936 (time 1770447843.77 total 119.84 last 293.54) - /unlink 35512 (time 1770447853.78 total 129.84 last 257.52) - /unlink 38145 (time 1770447863.78 total 139.85 last 263.29) - /unlink 40000 (time 1770447870.36 total 146.43 last 281.86) - /unlink 43071 (time 1770447880.36 total 156.43 last 307.05) - /unlink 46145 (time 1770447890.36 total 166.43 last 307.35) - /unlink 48953 (time 1770447900.36 total 176.43 last 280.79) - /unlink 50000 (time 1770447903.96 total 180.03 last 290.85) - /unlink 52707 (time 1770447913.97 total 190.03 last 270.60) - /unlink 55106 (time 1770447923.97 total 200.04 last 239.85) - /unlink 57678 (time 1770447933.97 total 210.04 last 257.20) - /unlink 60000 (time 1770447941.97 total 218.04 last 290.02) - /unlink 62597 (time 1770447951.98 total 228.05 last 259.63) total: 64768 /unlink in 235.08 seconds: 275.51 ops/second Waiting for MDT destroys to complete oleg236-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 - open/close 1493 (time 1770447983.32 total 10.00 last 149.25) - open/close 3199 (time 1770447993.32 total 20.01 last 170.55) - open/close 4878 (time 1770448003.32 total 30.01 last 167.87) - open/close 6541 (time 1770448013.32 total 40.01 last 166.28) - open/close 8024 (time 1770448023.32 total 50.01 last 148.27) - open/close 9628 (time 1770448033.32 total 60.01 last 160.40) - open/close 10000 (time 1770448035.49 total 62.18 last 171.49) - open/close 11621 (time 1770448045.50 total 72.18 last 162.05) - open/close 13249 (time 1770448055.50 total 82.19 last 162.72) - open/close 14999 (time 1770448065.50 total 92.19 last 174.92) - open/close 17148 (time 1770448075.51 total 102.20 last 214.76) - open/close 19781 (time 1770448085.52 total 112.21 last 263.07) - open/close 21961 (time 1770448095.53 total 122.21 last 217.88) - open/close 24646 (time 1770448105.53 total 132.22 last 268.43) - open/close 26746 (time 1770448115.53 total 142.22 last 209.94) - open/close 28918 (time 1770448125.54 total 152.22 last 217.10) - open/close 30000 (time 1770448130.07 total 156.76 last 238.46) - open/close 32155 (time 1770448140.07 total 166.76 last 215.49) - open/close 34487 (time 1770448150.07 total 176.76 last 233.18) - open/close 36959 (time 1770448160.08 total 186.77 last 247.12) - open/close 39451 (time 1770448170.08 total 196.77 last 249.19) - open/close 40000 (time 1770448172.18 total 198.87 last 260.64) - open/close 43098 (time 1770448182.19 total 208.88 last 309.70) - open/close 45741 (time 1770448192.19 total 218.88 last 264.20) - open/close 48695 (time 1770448202.20 total 228.88 last 295.28) - open/close 50000 (time 1770448208.26 total 234.95 last 215.15) - open/close 52515 (time 1770448218.27 total 244.95 last 251.37) - open/close 55278 (time 1770448228.27 total 254.95 last 276.29) - open/close 57474 (time 1770448238.27 total 264.96 last 219.52) - open/close 60000 (time 1770448247.52 total 274.21 last 273.14) - open/close 63007 (time 1770448257.52 total 284.21 last 300.64) total: 64768 open/close in 290.30 seconds: 223.11 ops/second - /unlink 5639 (time 1770448275.16 total 10.00 last 563.88) - /unlink 10000 (time 1770448283.85 total 18.69 last 501.80) - /unlink 14974 (time 1770448293.86 total 28.70 last 496.87) - /unlink 20000 (time 1770448303.83 total 38.68 last 503.94) - /unlink 25224 (time 1770448313.83 total 48.68 last 522.38) - /unlink 30000 (time 1770448322.37 total 57.21 last 559.67) - /unlink 36692 (time 1770448332.37 total 67.21 last 669.12) - /unlink 40000 (time 1770448336.72 total 71.56 last 759.95) - /unlink 46004 (time 1770448346.74 total 81.58 last 599.37) - /unlink 50000 (time 1770448352.62 total 87.46 last 679.37) - /unlink 56252 (time 1770448362.62 total 97.46 last 625.20) - /unlink 60000 (time 1770448367.87 total 102.71 last 714.29) total: 64768 /unlink in 109.23 seconds: 592.94 ops/second Waiting for MDT destroys to complete oleg236-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 - open/close 2837 (time 1770448393.17 total 10.00 last 283.67) - open/close 6711 (time 1770448403.17 total 20.00 last 387.34) - open/close 10000 (time 1770448410.05 total 26.88 last 477.89) - open/close 13950 (time 1770448420.06 total 36.89 last 394.91) - open/close 17360 (time 1770448430.06 total 46.89 last 340.95) - open/close 20000 (time 1770448437.20 total 54.03 last 369.48) - open/close 24442 (time 1770448447.20 total 64.03 last 444.17) - open/close 29102 (time 1770448457.20 total 74.04 last 465.96) - open/close 33829 (time 1770448467.20 total 84.04 last 472.67) - open/close 38330 (time 1770448477.21 total 94.04 last 450.06) - open/close 40000 (time 1770448481.29 total 98.13 last 408.47) - open/close 44380 (time 1770448491.29 total 108.13 last 437.97) - open/close 48920 (time 1770448501.30 total 118.13 last 453.94) - open/close 50000 (time 1770448503.97 total 120.80 last 403.39) - open/close 54382 (time 1770448513.97 total 130.81 last 438.15) - open/close 58941 (time 1770448523.98 total 140.81 last 455.84) - open/close 60000 (time 1770448526.46 total 143.29 last 426.93) - open/close 64732 (time 1770448536.46 total 153.29 last 473.18) total: 64768 open/close in 153.35 seconds: 422.35 ops/second - /unlink 8816 (time 1770448547.83 total 10.00 last 881.58) - /unlink 17924 (time 1770448557.83 total 20.00 last 910.79) - /unlink 20000 (time 1770448560.19 total 22.36 last 878.86) - /unlink 25461 (time 1770448570.19 total 32.36 last 546.07) - /unlink 27043 (time 1770448580.19 total 42.37 last 158.14) - /unlink 28699 (time 1770448590.20 total 52.37 last 165.50) - /unlink 30000 (time 1770448599.68 total 61.85 last 137.22) - /unlink 31368 (time 1770448609.68 total 71.86 last 136.77) - /unlink 33277 (time 1770448619.70 total 81.87 last 190.64) - /unlink 35560 (time 1770448629.70 total 91.87 last 228.26) - /unlink 37841 (time 1770448639.70 total 101.87 last 228.07) - /unlink 40000 (time 1770448648.58 total 110.75 last 243.19) - /unlink 42986 (time 1770448658.58 total 120.75 last 298.45) - /unlink 45725 (time 1770448668.58 total 130.76 last 273.89) - /unlink 48571 (time 1770448678.59 total 140.76 last 284.50) - /unlink 50000 (time 1770448684.13 total 146.30 last 257.88) - /unlink 52793 (time 1770448694.13 total 156.30 last 279.27) - /unlink 55744 (time 1770448704.13 total 166.30 last 295.06) - /unlink 58598 (time 1770448714.13 total 176.30 last 285.34) - /unlink 60000 (time 1770448719.04 total 181.21 last 285.69) - /unlink 62822 (time 1770448729.04 total 191.22 last 282.11) total: 64768 /unlink in 199.06 seconds: 325.36 ops/second Waiting for MDT destroys to complete oleg236-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 - open/close 1561 (time 1770448755.48 total 10.00 last 156.08) - open/close 2912 (time 1770448765.49 total 20.01 last 135.02) - open/close 4511 (time 1770448775.49 total 30.01 last 159.87) - open/close 5931 (time 1770448785.49 total 40.01 last 141.93) - open/close 7247 (time 1770448795.50 total 50.02 last 131.48) - open/close 8633 (time 1770448805.51 total 60.03 last 138.57) - open/close 9999 (time 1770448815.51 total 70.03 last 136.54) - open/close 11595 (time 1770448825.51 total 80.03 last 159.59) - open/close 13192 (time 1770448835.51 total 90.03 last 159.67) - open/close 14492 (time 1770448845.52 total 100.04 last 129.92) - open/close 16272 (time 1770448855.52 total 110.04 last 177.91) - open/close 17883 (time 1770448865.53 total 120.05 last 161.02) - open/close 19124 (time 1770448875.54 total 130.06 last 123.96) - open/close 20000 (time 1770448882.34 total 136.86 last 128.82) - open/close 21342 (time 1770448892.34 total 146.86 last 134.16) - open/close 22715 (time 1770448902.35 total 156.87 last 137.25) - open/close 24504 (time 1770448912.35 total 166.87 last 178.88) - open/close 26058 (time 1770448922.35 total 176.87 last 155.38) - open/close 27545 (time 1770448932.35 total 186.87 last 148.64) - open/close 29376 (time 1770448942.36 total 196.88 last 183.04) - open/close 30000 (time 1770448945.95 total 200.47 last 173.79) - open/close 31490 (time 1770448955.96 total 210.48 last 148.81) - open/close 33006 (time 1770448965.96 total 220.48 last 151.59) - open/close 34300 (time 1770448975.96 total 230.48 last 129.39) - open/close 35465 (time 1770448985.96 total 240.48 last 116.47) - open/close 36549 (time 1770448995.97 total 250.49 last 108.38) - open/close 37639 (time 1770449005.97 total 260.49 last 108.95) - open/close 38872 (time 1770449015.98 total 270.50 last 123.20) - open/close 39980 (time 1770449025.98 total 280.50 last 110.77) - open/close 41144 (time 1770449035.98 total 290.50 last 116.38) - open/close 42586 (time 1770449045.98 total 300.50 last 144.19) - open/close 43783 (time 1770449055.99 total 310.51 last 119.67) - open/close 44974 (time 1770449065.99 total 320.51 last 119.01) - open/close 46203 (time 1770449075.99 total 330.51 last 122.89) - open/close 47475 (time 1770449086.00 total 340.52 last 127.15) - open/close 48679 (time 1770449096.00 total 350.52 last 120.40) - open/close 50000 (time 1770449105.55 total 360.07 last 138.37) - open/close 51690 (time 1770449115.55 total 370.07 last 168.98) - open/close 53903 (time 1770449125.55 total 380.07 last 221.18) - open/close 55289 (time 1770449135.55 total 390.07 last 138.57) - open/close 56943 (time 1770449145.56 total 400.08 last 165.35) - open/close 58479 (time 1770449155.56 total 410.08 last 153.58) - open/close 59855 (time 1770449165.56 total 420.08 last 137.59) - open/close 61382 (time 1770449175.56 total 430.08 last 152.68) - open/close 62727 (time 1770449185.56 total 440.08 last 134.46) - open/close 64227 (time 1770449195.57 total 450.09 last 149.92) total: 64768 open/close in 453.24 seconds: 142.90 ops/second - /unlink 3764 (time 1770449210.69 total 10.00 last 376.34) - /unlink 7264 (time 1770449220.69 total 20.00 last 349.99) - /unlink 10000 (time 1770449228.87 total 28.19 last 334.25) - /unlink 13128 (time 1770449238.88 total 38.19 last 312.68) - /unlink 15777 (time 1770449248.88 total 48.19 last 264.86) - /unlink 19254 (time 1770449258.88 total 58.19 last 347.70) - /unlink 20000 (time 1770449261.44 total 60.75 last 291.62) - /unlink 22872 (time 1770449271.44 total 70.75 last 287.14) - /unlink 26446 (time 1770449281.44 total 80.75 last 357.36) - /unlink 29708 (time 1770449291.45 total 90.76 last 325.93) - /unlink 34222 (time 1770449301.45 total 100.76 last 451.28) - /unlink 38752 (time 1770449311.45 total 110.76 last 453.00) - /unlink 40000 (time 1770449314.12 total 113.43 last 467.74) - /unlink 43590 (time 1770449324.13 total 123.44 last 358.82) - /unlink 46652 (time 1770449334.13 total 133.44 last 306.09) - /unlink 49331 (time 1770449344.13 total 143.44 last 267.89) - /unlink 50000 (time 1770449346.83 total 146.15 last 247.46) - /unlink 52254 (time 1770449356.83 total 156.15 last 225.40) - /unlink 55442 (time 1770449366.84 total 166.15 last 318.69) - /unlink 58825 (time 1770449376.84 total 176.16 last 338.07) - /unlink 60000 (time 1770449380.77 total 180.08 last 299.41) - /unlink 62962 (time 1770449390.77 total 190.08 last 296.18) total: 64768 /unlink in 195.11 seconds: 331.96 ops/second Waiting for MDT destroys to complete oleg236-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg236-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg236-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg236-server stop mds service on oleg236-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg236-server stop mds service on oleg236-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg236-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing unload_modules_local oleg236-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg236-client: oleg236-client: ssh exited with exit code 2 pdsh@oleg236-client: oleg236-server: ssh exited with exit code 2 pdsh@oleg236-client: oleg236-client: ssh exited with exit code 2 pdsh@oleg236-client: oleg236-server: ssh exited with exit code 2