-----============= acceptance-small: conf-sanity ============----- Fri Oct 24 01:41:54 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg121-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball Stopping clients: oleg121-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg121-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg121-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg121-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg121-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg121-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg121-server oleg121-server: oleg121-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg121-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg121-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg121-server: oleg121-server.virtnet: executing set_default_debug -1 all pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg121-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg121-server: oleg121-server.virtnet: executing set_default_debug -1 all pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg121-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg121-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg121-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg121-server: oleg121-server.virtnet: executing set_default_debug -1 all pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg121-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg121-server: oleg121-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg121-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg121-server: oleg121-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg121-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg121-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg121-server stop mds service on oleg121-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg121-server stop mds service on oleg121-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg121-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 01:44:29 (1761284669) Stopping clients: oleg121-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg121-client.virtnet /mnt/lustre2 (opts:-f) oleg121-server: oleg121-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg121-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg121-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg121-server: oleg121-server.virtnet: executing set_default_debug -1 all pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg121-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg121-server: oleg121-server.virtnet: executing set_default_debug -1 all pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg121-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg121-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg121-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg121-server: oleg121-server.virtnet: executing set_default_debug -1 all pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg121-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 279 79713 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 257 79735 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65976 536 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 713 (time 1761284753.03 total 10.01 last 71.25) - open/close 1502 (time 1761284763.04 total 20.02 last 78.82) - open/close 2405 (time 1761284773.05 total 30.02 last 90.25) - open/close 3286 (time 1761284783.05 total 40.03 last 88.05) - open/close 4174 (time 1761284793.06 total 50.03 last 88.74) - open/close 5072 (time 1761284803.07 total 60.04 last 89.71) - open/close 5927 (time 1761284813.08 total 70.05 last 85.42) - open/close 6783 (time 1761284823.08 total 80.06 last 85.59) - open/close 7643 (time 1761284833.08 total 90.06 last 85.99) - open/close 8568 (time 1761284843.09 total 100.07 last 92.41) - open/close 9508 (time 1761284853.10 total 110.07 last 93.94) - open/close 10000 (time 1761284858.84 total 115.82 last 85.59) - open/close 10880 (time 1761284868.85 total 125.83 last 87.97) - open/close 11763 (time 1761284878.86 total 135.83 last 88.24) - open/close 12640 (time 1761284888.86 total 145.84 last 87.65) - open/close 13532 (time 1761284898.87 total 155.84 last 89.14) - open/close 14322 (time 1761284908.87 total 165.85 last 78.94) - open/close 15214 (time 1761284918.88 total 175.86 last 89.15) - open/close 16103 (time 1761284928.89 total 185.87 last 88.80) - open/close 16975 (time 1761284938.90 total 195.87 last 87.15) - open/close 17831 (time 1761284948.91 total 205.88 last 85.51) - open/close 18728 (time 1761284958.91 total 215.89 last 89.64) - open/close 19666 (time 1761284968.92 total 225.90 last 93.75) - open/close 20000 (time 1761284972.57 total 229.55 last 91.50) - open/close 20846 (time 1761284982.57 total 239.55 last 84.58) - open/close 21740 (time 1761284992.58 total 249.55 last 89.38) - open/close 22616 (time 1761285002.58 total 259.56 last 87.52) - open/close 23465 (time 1761285012.59 total 269.56 last 84.89) - open/close 24363 (time 1761285022.59 total 279.57 last 89.76) - open/close 25337 (time 1761285032.60 total 289.57 last 97.33) - open/close 26217 (time 1761285042.60 total 299.57 last 87.99) - open/close 27094 (time 1761285052.61 total 309.59 last 87.60) - open/close 27903 (time 1761285062.61 total 319.59 last 80.85) - open/close 28766 (time 1761285072.62 total 329.59 last 86.28) - open/close 29612 (time 1761285082.63 total 339.61 last 84.45) - open/close 30000 (time 1761285087.19 total 344.16 last 85.25) - open/close 30878 (time 1761285097.20 total 354.18 last 87.68) - open/close 31773 (time 1761285107.21 total 364.18 last 89.44) - open/close 32640 (time 1761285117.21 total 374.19 last 86.64) - open/close 33480 (time 1761285127.23 total 384.21 last 83.86) - open/close 34362 (time 1761285137.24 total 394.21 last 88.15) - open/close 35228 (time 1761285147.24 total 404.21 last 86.59) - open/close 36128 (time 1761285157.25 total 414.23 last 89.85) - open/close 37005 (time 1761285167.25 total 424.23 last 87.68) - open/close 37918 (time 1761285177.26 total 434.23 last 91.30) - open/close 38932 (time 1761285187.26 total 444.23 last 101.37) - open/close 39833 (time 1761285197.27 total 454.24 last 90.03) - open/close 40731 (time 1761285207.28 total 464.26 last 89.68) - open/close 41620 (time 1761285217.28 total 474.26 last 88.86) - open/close 42425 (time 1761285227.31 total 484.28 last 80.32) - open/close 43280 (time 1761285237.32 total 494.30 last 85.38) - open/close 44114 (time 1761285247.33 total 504.31 last 83.33) - open/close 45026 (time 1761285257.33 total 514.31 last 91.18) - open/close 45916 (time 1761285267.35 total 524.32 last 88.88) - open/close 46792 (time 1761285277.35 total 534.33 last 87.55) - open/close 47706 (time 1761285287.35 total 544.33 last 91.40) - open/close 48571 (time 1761285297.36 total 554.34 last 86.44) - open/close 49441 (time 1761285307.37 total 564.35 last 86.91) - open/close 50000 (time 1761285313.90 total 570.87 last 85.64) - open/close 50803 (time 1761285323.91 total 580.88 last 80.23) - open/close 51644 (time 1761285333.91 total 590.88 last 84.09) - open/close 52435 (time 1761285343.92 total 600.89 last 79.03) - open/close 53246 (time 1761285353.92 total 610.90 last 81.04) - open/close 54080 (time 1761285363.93 total 620.90 last 83.36) - open/close 54886 (time 1761285373.93 total 630.90 last 80.60) - open/close 55728 (time 1761285383.93 total 640.91 last 84.16) - open/close 56562 (time 1761285393.94 total 650.91 last 83.37) - open/close 57363 (time 1761285403.96 total 660.93 last 79.94) - open/close 58147 (time 1761285413.96 total 670.94 last 78.36) - open/close 58908 (time 1761285423.97 total 680.95 last 76.02) - open/close 59622 (time 1761285433.98 total 690.96 last 71.33) - open/close 60000 (time 1761285439.03 total 696.01 last 74.82) - open/close 60736 (time 1761285449.03 total 706.01 last 73.60) - open/close 61506 (time 1761285459.04 total 716.02 last 76.92) - open/close 62262 (time 1761285469.05 total 726.02 last 75.58) - open/close 63071 (time 1761285479.05 total 736.03 last 80.84) - open/close 63887 (time 1761285489.06 total 746.03 last 81.57) total: 64768 open/close in 755.90 seconds: 85.68 ops/second - /unlink 936 (time 1761285511.41 total 10.01 last 93.52) - /unlink 2104 (time 1761285521.41 total 20.01 last 116.80) - /unlink 3055 (time 1761285531.42 total 30.02 last 95.02) - /unlink 3914 (time 1761285541.43 total 40.03 last 85.83) - /unlink 4766 (time 1761285551.45 total 50.05 last 85.01) - /unlink 5742 (time 1761285561.45 total 60.05 last 97.59) - /unlink 6610 (time 1761285571.46 total 70.05 last 86.74) - /unlink 7702 (time 1761285581.46 total 80.06 last 109.14) - /unlink 8611 (time 1761285591.48 total 90.07 last 90.79) - /unlink 9505 (time 1761285601.49 total 100.09 last 89.28) - /unlink 10000 (time 1761285606.71 total 105.31 last 94.79) - /unlink 10882 (time 1761285616.71 total 115.31 last 88.17) - /unlink 11787 (time 1761285626.72 total 125.31 last 90.48) - /unlink 12719 (time 1761285636.73 total 135.32 last 93.10) - /unlink 13663 (time 1761285646.73 total 145.33 last 94.36) - /unlink 14581 (time 1761285656.74 total 155.33 last 91.75) - /unlink 15419 (time 1761285666.75 total 165.34 last 83.72) - /unlink 16253 (time 1761285676.75 total 175.35 last 83.36) - /unlink 17059 (time 1761285686.76 total 185.35 last 80.56) - /unlink 17847 (time 1761285696.77 total 195.37 last 78.67) - /unlink 18686 (time 1761285706.78 total 205.38 last 83.82) - /unlink 19508 (time 1761285716.79 total 215.39 last 82.12) - /unlink 20000 (time 1761285722.79 total 221.39 last 82.02) - /unlink 20731 (time 1761285732.80 total 231.40 last 73.02) - /unlink 21560 (time 1761285742.81 total 241.40 last 82.84) - /unlink 22389 (time 1761285752.82 total 251.42 last 82.77) - /unlink 23247 (time 1761285762.83 total 261.43 last 85.74) - /unlink 24201 (time 1761285772.83 total 271.43 last 95.36) - /unlink 25015 (time 1761285782.84 total 281.44 last 81.33) - /unlink 25808 (time 1761285792.84 total 291.44 last 79.28) - /unlink 26577 (time 1761285802.85 total 301.45 last 76.85) - /unlink 27375 (time 1761285812.86 total 311.45 last 79.75) - /unlink 28210 (time 1761285822.86 total 321.46 last 83.48) - /unlink 29155 (time 1761285832.86 total 331.46 last 94.49) - /unlink 30000 (time 1761285841.94 total 340.53 last 93.08) - /unlink 30821 (time 1761285851.94 total 350.54 last 82.09) - /unlink 31679 (time 1761285861.94 total 360.54 last 85.75) - /unlink 32563 (time 1761285871.95 total 370.55 last 88.34) - /unlink 33509 (time 1761285881.95 total 380.55 last 94.58) - /unlink 34477 (time 1761285891.96 total 390.55 last 96.77) - /unlink 35336 (time 1761285901.96 total 400.55 last 85.88) - /unlink 36206 (time 1761285911.97 total 410.56 last 86.93) - /unlink 37048 (time 1761285921.97 total 420.57 last 84.17) - /unlink 37869 (time 1761285931.97 total 430.57 last 82.07) - /unlink 38709 (time 1761285942.00 total 440.59 last 83.80) - /unlink 39628 (time 1761285952.00 total 450.60 last 91.85) - /unlink 40000 (time 1761285956.52 total 455.12 last 82.29) - /unlink 40854 (time 1761285966.52 total 465.12 last 85.39) - /unlink 41733 (time 1761285976.53 total 475.13 last 87.83) - /unlink 42669 (time 1761285986.54 total 485.13 last 93.57) - /unlink 43566 (time 1761285996.54 total 495.13 last 89.68) - /unlink 44509 (time 1761286006.54 total 505.14 last 94.24) - /unlink 45345 (time 1761286016.55 total 515.15 last 83.54) - /unlink 46187 (time 1761286026.55 total 525.15 last 84.18) - /unlink 47048 (time 1761286036.56 total 535.16 last 86.04) - /unlink 47996 (time 1761286046.57 total 545.17 last 94.70) - /unlink 48879 (time 1761286056.58 total 555.17 last 88.24) - /unlink 49703 (time 1761286066.59 total 565.18 last 82.32) - /unlink 50000 (time 1761286070.04 total 568.63 last 86.11) - /unlink 50908 (time 1761286080.04 total 578.64 last 90.78) - /unlink 51780 (time 1761286090.04 total 588.64 last 87.17) - /unlink 52588 (time 1761286100.05 total 598.65 last 80.73) - /unlink 53463 (time 1761286110.05 total 608.65 last 87.46) - /unlink 54439 (time 1761286120.06 total 618.66 last 97.56) - /unlink 55387 (time 1761286130.07 total 628.67 last 94.69) - /unlink 56326 (time 1761286140.08 total 638.68 last 93.81) - /unlink 57276 (time 1761286150.08 total 648.68 last 94.99) - /unlink 58173 (time 1761286160.09 total 658.68 last 89.64) - /unlink 59052 (time 1761286170.09 total 668.69 last 87.84) - /unlink 59923 (time 1761286180.11 total 678.70 last 86.99) - /unlink 60805 (time 1761286190.12 total 688.71 last 88.10) - /unlink 61658 (time 1761286200.13 total 698.72 last 85.22) - /unlink 62558 (time 1761286210.13 total 708.73 last 89.96) - /unlink 63464 (time 1761286220.13 total 718.73 last 90.59) - /unlink 64436 (time 1761286230.14 total 728.73 last 97.16) total: 64768 /unlink in 732.45 seconds: 88.43 ops/second Waiting for MDT destroys to complete oleg121-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 - open/close 888 (time 1761286253.86 total 10.01 last 88.72) - open/close 1774 (time 1761286263.87 total 20.02 last 88.50) - open/close 2625 (time 1761286273.88 total 30.03 last 85.03) - open/close 3510 (time 1761286283.88 total 40.04 last 88.43) - open/close 4350 (time 1761286293.89 total 50.04 last 83.99) - open/close 5243 (time 1761286303.89 total 60.04 last 89.27) - open/close 6055 (time 1761286313.89 total 70.05 last 81.17) - open/close 6879 (time 1761286323.90 total 80.05 last 82.37) - open/close 7678 (time 1761286333.90 total 90.05 last 79.90) - open/close 8471 (time 1761286343.90 total 100.05 last 79.26) - open/close 9277 (time 1761286353.92 total 110.07 last 80.49) - open/close 10000 (time 1761286362.51 total 118.66 last 84.11) - open/close 10791 (time 1761286372.52 total 128.67 last 79.06) - open/close 11677 (time 1761286382.52 total 138.67 last 88.57) - open/close 12464 (time 1761286392.52 total 148.68 last 78.68) - open/close 13265 (time 1761286402.53 total 158.68 last 80.04) - open/close 14009 (time 1761286412.54 total 168.69 last 74.37) - open/close 14680 (time 1761286422.54 total 178.69 last 67.07) - open/close 15430 (time 1761286432.55 total 188.70 last 74.91) - open/close 16169 (time 1761286442.58 total 198.73 last 73.71) - open/close 16985 (time 1761286452.59 total 208.74 last 81.51) - open/close 17872 (time 1761286462.59 total 218.74 last 88.67) - open/close 18728 (time 1761286472.60 total 228.76 last 85.49) - open/close 19622 (time 1761286482.61 total 238.77 last 89.32) - open/close 20000 (time 1761286486.97 total 243.13 last 86.68) - open/close 20761 (time 1761286496.98 total 253.13 last 76.05) - open/close 21577 (time 1761286506.98 total 263.13 last 81.60) - open/close 22360 (time 1761286517.00 total 273.15 last 78.14) - open/close 23187 (time 1761286527.01 total 283.16 last 82.65) - open/close 24016 (time 1761286537.03 total 293.18 last 82.76) - open/close 24825 (time 1761286547.03 total 303.18 last 80.89) - open/close 25650 (time 1761286557.03 total 313.18 last 82.49) - open/close 26461 (time 1761286567.05 total 323.20 last 80.94) - open/close 27299 (time 1761286577.05 total 333.21 last 83.75) - open/close 28145 (time 1761286587.06 total 343.21 last 84.53) - open/close 28953 (time 1761286597.08 total 353.23 last 80.68) - open/close 29745 (time 1761286607.08 total 363.24 last 79.15) - open/close 30000 (time 1761286610.05 total 366.20 last 86.01) - open/close 30813 (time 1761286620.05 total 376.20 last 81.30) - open/close 31640 (time 1761286630.05 total 386.20 last 82.66) - open/close 32481 (time 1761286640.06 total 396.21 last 84.04) - open/close 33197 (time 1761286650.07 total 406.23 last 71.51) - open/close 33877 (time 1761286660.08 total 416.23 last 67.98) - open/close 34622 (time 1761286670.09 total 426.24 last 74.41) - open/close 35357 (time 1761286680.10 total 436.25 last 73.41) - open/close 36095 (time 1761286690.11 total 446.26 last 73.73) - open/close 36888 (time 1761286700.11 total 456.26 last 79.29) - open/close 37578 (time 1761286710.13 total 466.28 last 68.87) - open/close 38270 (time 1761286720.14 total 476.29 last 69.14) - open/close 38996 (time 1761286730.14 total 486.30 last 72.56) - open/close 39704 (time 1761286740.15 total 496.30 last 70.75) - open/close 40000 (time 1761286744.02 total 500.17 last 76.54) - open/close 40785 (time 1761286754.02 total 510.17 last 78.49) - open/close 41565 (time 1761286764.03 total 520.18 last 77.95) - open/close 42275 (time 1761286774.03 total 530.18 last 70.98) - open/close 43128 (time 1761286784.04 total 540.19 last 85.24) - open/close 43909 (time 1761286794.05 total 550.20 last 78.00) - open/close 44726 (time 1761286804.06 total 560.22 last 81.57) - open/close 45455 (time 1761286814.07 total 570.22 last 72.87) - open/close 46207 (time 1761286824.08 total 580.24 last 75.10) - open/close 47021 (time 1761286834.09 total 590.24 last 81.35) - open/close 47757 (time 1761286844.10 total 600.25 last 73.55) - open/close 48480 (time 1761286854.10 total 610.25 last 72.28) - open/close 49242 (time 1761286864.10 total 620.25 last 76.17) - open/close 49897 (time 1761286874.12 total 630.28 last 65.37) - open/close 50000 (time 1761286876.29 total 632.44 last 47.55) - open/close 50684 (time 1761286886.30 total 642.45 last 68.32) - open/close 51339 (time 1761286896.31 total 652.46 last 65.47) - open/close 52046 (time 1761286906.34 total 662.49 last 70.45) - open/close 52694 (time 1761286916.37 total 672.52 last 64.64) - open/close 53471 (time 1761286926.38 total 682.53 last 77.60) - open/close 54197 (time 1761286936.39 total 692.54 last 72.55) - open/close 54935 (time 1761286946.39 total 702.55 last 73.73) - open/close 55665 (time 1761286956.39 total 712.55 last 73.00) - open/close 56382 (time 1761286966.40 total 722.55 last 71.66) - open/close 57019 (time 1761286976.40 total 732.56 last 63.68) - open/close 57740 (time 1761286986.41 total 742.56 last 72.04) - open/close 58523 (time 1761286996.42 total 752.57 last 78.27) - open/close 59289 (time 1761287006.42 total 762.57 last 76.55) - open/close 60000 (time 1761287016.05 total 772.20 last 73.87) - open/close 60746 (time 1761287026.05 total 782.21 last 74.56) - open/close 61481 (time 1761287036.06 total 792.21 last 73.47) - open/close 62223 (time 1761287046.07 total 802.22 last 74.11) - open/close 63011 (time 1761287056.07 total 812.22 last 78.79) - open/close 63654 (time 1761287066.08 total 822.23 last 64.25) - open/close 64363 (time 1761287076.08 total 832.23 last 70.88) total: 64768 open/close in 837.23 seconds: 77.36 ops/second - /unlink 779 (time 1761287095.27 total 10.01 last 77.86) - /unlink 1565 (time 1761287105.28 total 20.02 last 78.49) - /unlink 2328 (time 1761287115.29 total 30.03 last 76.23) - /unlink 3144 (time 1761287125.29 total 40.03 last 81.57) - /unlink 3843 (time 1761287135.31 total 50.04 last 69.83) - /unlink 4497 (time 1761287145.31 total 60.05 last 65.34) - /unlink 5218 (time 1761287155.33 total 70.07 last 71.97) - /unlink 5953 (time 1761287165.34 total 80.08 last 73.46) - /unlink 6745 (time 1761287175.34 total 90.08 last 79.20) - /unlink 7424 (time 1761287185.35 total 100.09 last 67.80) - /unlink 8130 (time 1761287195.36 total 110.10 last 70.53) - /unlink 8831 (time 1761287205.37 total 120.10 last 70.08) - /unlink 9521 (time 1761287215.37 total 130.11 last 68.96) - /unlink 10000 (time 1761287222.10 total 136.84 last 71.15) - /unlink 10790 (time 1761287232.11 total 146.85 last 78.95) - /unlink 11583 (time 1761287242.11 total 156.85 last 79.26) - /unlink 12357 (time 1761287252.12 total 166.86 last 77.34) - /unlink 13187 (time 1761287262.13 total 176.87 last 82.92) - /unlink 14066 (time 1761287272.14 total 186.88 last 87.83) - /unlink 14853 (time 1761287282.15 total 196.88 last 78.64) - /unlink 15548 (time 1761287292.16 total 206.89 last 69.44) - /unlink 16328 (time 1761287302.16 total 216.89 last 77.99) - /unlink 17203 (time 1761287312.16 total 226.90 last 87.46) - /unlink 17984 (time 1761287322.17 total 236.91 last 78.04) - /unlink 18761 (time 1761287332.17 total 246.91 last 77.70) - /unlink 19563 (time 1761287342.18 total 256.91 last 80.14) - /unlink 20000 (time 1761287348.53 total 263.27 last 68.81) - /unlink 20736 (time 1761287358.54 total 273.27 last 73.54) - /unlink 21437 (time 1761287368.54 total 283.28 last 70.08) - /unlink 22212 (time 1761287378.54 total 293.28 last 77.48) - /unlink 22959 (time 1761287388.55 total 303.28 last 74.67) - /unlink 23728 (time 1761287398.56 total 313.29 last 76.82) - /unlink 24455 (time 1761287408.56 total 323.30 last 72.64) - /unlink 25182 (time 1761287418.57 total 333.31 last 72.67) - /unlink 25887 (time 1761287428.58 total 343.31 last 70.45) - /unlink 26552 (time 1761287438.59 total 353.33 last 66.42) - /unlink 27277 (time 1761287448.60 total 363.33 last 72.45) - /unlink 28004 (time 1761287458.61 total 373.34 last 72.63) - /unlink 28848 (time 1761287468.61 total 383.35 last 84.34) - /unlink 29680 (time 1761287478.61 total 393.35 last 83.18) - /unlink 30000 (time 1761287482.40 total 397.14 last 84.59) - /unlink 30868 (time 1761287492.40 total 407.14 last 86.77) - /unlink 31636 (time 1761287502.41 total 417.15 last 76.70) - /unlink 32478 (time 1761287512.41 total 427.15 last 84.20) - /unlink 33275 (time 1761287522.42 total 437.16 last 79.63) - /unlink 34109 (time 1761287532.43 total 447.16 last 83.37) - /unlink 34918 (time 1761287542.43 total 457.17 last 80.84) - /unlink 35696 (time 1761287552.44 total 467.18 last 77.75) - /unlink 36518 (time 1761287562.44 total 477.18 last 82.19) - /unlink 37346 (time 1761287572.44 total 487.18 last 82.77) - /unlink 38202 (time 1761287582.45 total 497.18 last 85.58) - /unlink 39050 (time 1761287592.45 total 507.19 last 84.77) - /unlink 39858 (time 1761287602.45 total 517.19 last 80.76) - /unlink 40742 (time 1761287612.46 total 527.20 last 88.31) - /unlink 41585 (time 1761287622.49 total 537.23 last 84.09) - /unlink 42473 (time 1761287632.50 total 547.24 last 88.73) - /unlink 43303 (time 1761287642.51 total 557.25 last 82.91) - /unlink 44163 (time 1761287652.51 total 567.25 last 85.97) - /unlink 44979 (time 1761287662.52 total 577.25 last 81.57) - /unlink 45859 (time 1761287672.53 total 587.26 last 87.91) - /unlink 46776 (time 1761287682.53 total 597.27 last 91.66) - /unlink 47688 (time 1761287692.53 total 607.27 last 91.17) - /unlink 48479 (time 1761287702.54 total 617.27 last 79.08) - /unlink 49342 (time 1761287712.54 total 627.27 last 86.30) - /unlink 50000 (time 1761287721.11 total 635.85 last 76.74) - /unlink 50756 (time 1761287731.12 total 645.86 last 75.51) - /unlink 51626 (time 1761287741.13 total 655.87 last 86.96) - /unlink 52465 (time 1761287751.13 total 665.87 last 83.89) - /unlink 53313 (time 1761287761.14 total 675.88 last 84.69) - /unlink 54169 (time 1761287771.15 total 685.89 last 85.56) - /unlink 55033 (time 1761287781.16 total 695.90 last 86.30) - /unlink 55930 (time 1761287791.16 total 705.90 last 89.69) - /unlink 56591 (time 1761287801.16 total 715.90 last 66.07) - /unlink 57467 (time 1761287811.17 total 725.90 last 87.58) - /unlink 58342 (time 1761287821.17 total 735.91 last 87.45) - /unlink 59092 (time 1761287831.18 total 745.91 last 74.98) - /unlink 59890 (time 1761287841.19 total 755.93 last 79.65) - /unlink 60715 (time 1761287851.19 total 765.93 last 82.50) - /unlink 61501 (time 1761287861.20 total 775.93 last 78.58) - /unlink 62290 (time 1761287871.20 total 785.94 last 78.87) - /unlink 63131 (time 1761287881.20 total 795.94 last 84.08) - /unlink 63958 (time 1761287891.20 total 805.94 last 82.69) total: 64768 /unlink in 815.47 seconds: 79.42 ops/second Waiting for MDT destroys to complete oleg121-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 - open/close 880 (time 1761287929.36 total 10.01 last 87.93) - open/close 1576 (time 1761287939.36 total 20.01 last 69.60) - open/close 2191 (time 1761287949.36 total 30.01 last 61.48) - open/close 2942 (time 1761287959.37 total 40.02 last 75.06) - open/close 3801 (time 1761287969.38 total 50.02 last 85.84) - open/close 4668 (time 1761287979.38 total 60.03 last 86.64) - open/close 5563 (time 1761287989.39 total 70.04 last 89.46) - open/close 6452 (time 1761287999.39 total 80.04 last 88.85) - open/close 7321 (time 1761288009.39 total 90.04 last 86.90) - open/close 8192 (time 1761288019.40 total 100.05 last 87.05) - open/close 9053 (time 1761288029.41 total 110.05 last 86.04) - open/close 9953 (time 1761288039.42 total 120.07 last 89.89) - open/close 10817 (time 1761288049.43 total 130.08 last 86.30) - open/close 11679 (time 1761288059.43 total 140.08 last 86.17) - open/close 12562 (time 1761288069.44 total 150.09 last 88.25) - open/close 13402 (time 1761288079.44 total 160.09 last 83.97) - open/close 14285 (time 1761288089.44 total 170.09 last 88.28) - open/close 15090 (time 1761288099.45 total 180.10 last 80.43) - open/close 15850 (time 1761288109.46 total 190.11 last 75.95) - open/close 16788 (time 1761288119.47 total 200.12 last 93.71) - open/close 17685 (time 1761288129.48 total 210.13 last 89.60) - open/close 18506 (time 1761288139.48 total 220.13 last 82.08) - open/close 19399 (time 1761288149.49 total 230.14 last 89.26) - open/close 20000 (time 1761288156.19 total 236.84 last 89.62) - open/close 20914 (time 1761288166.19 total 246.84 last 91.39) - open/close 21779 (time 1761288176.20 total 256.85 last 86.45) - open/close 22697 (time 1761288186.20 total 266.85 last 91.79) - open/close 23627 (time 1761288196.21 total 276.86 last 92.93) - open/close 24592 (time 1761288206.22 total 286.87 last 96.38) - open/close 25500 (time 1761288216.23 total 296.88 last 90.67) - open/close 26437 (time 1761288226.24 total 306.88 last 93.69) - open/close 27425 (time 1761288236.24 total 316.88 last 98.80) - open/close 28349 (time 1761288246.24 total 326.89 last 92.36) - open/close 29250 (time 1761288256.25 total 336.90 last 90.00) - open/close 30000 (time 1761288264.38 total 345.03 last 92.22) - open/close 31004 (time 1761288274.39 total 355.04 last 100.32) - open/close 32076 (time 1761288284.40 total 365.05 last 107.14) - open/close 33142 (time 1761288294.40 total 375.05 last 106.58) - open/close 34137 (time 1761288304.41 total 385.06 last 99.38) - open/close 35166 (time 1761288314.42 total 395.07 last 102.83) - open/close 36229 (time 1761288324.42 total 405.07 last 106.26) - open/close 37257 (time 1761288334.44 total 415.09 last 102.57) - open/close 38287 (time 1761288344.46 total 425.11 last 102.87) - open/close 39364 (time 1761288354.46 total 435.11 last 107.64) - open/close 40000 (time 1761288360.53 total 441.18 last 104.73) - open/close 41036 (time 1761288370.55 total 451.20 last 103.48) - open/close 42032 (time 1761288380.56 total 461.21 last 99.49) - open/close 42810 (time 1761288390.57 total 471.22 last 77.68) - open/close 43409 (time 1761288400.59 total 481.23 last 59.82) - open/close 44009 (time 1761288410.60 total 491.25 last 59.93) - open/close 44505 (time 1761288420.60 total 501.25 last 49.59) - open/close 45200 (time 1761288430.61 total 511.26 last 69.45) - open/close 45920 (time 1761288440.61 total 521.26 last 71.96) - open/close 46748 (time 1761288450.62 total 531.27 last 82.75) - open/close 47666 (time 1761288460.62 total 541.27 last 91.76) - open/close 48548 (time 1761288470.62 total 551.27 last 88.19) - open/close 49465 (time 1761288480.63 total 561.27 last 91.69) - open/close 50000 (time 1761288486.77 total 567.42 last 87.10) - open/close 50872 (time 1761288496.77 total 577.42 last 87.20) - open/close 51727 (time 1761288506.78 total 587.43 last 85.41) - open/close 52579 (time 1761288516.78 total 597.43 last 85.19) - open/close 53403 (time 1761288526.79 total 607.44 last 82.33) - open/close 54298 (time 1761288536.79 total 617.44 last 89.45) - open/close 55157 (time 1761288546.80 total 627.45 last 85.83) - open/close 56029 (time 1761288556.81 total 637.46 last 87.16) - open/close 56836 (time 1761288566.81 total 647.46 last 80.67) - open/close 57669 (time 1761288576.82 total 657.47 last 83.23) - open/close 58529 (time 1761288586.83 total 667.48 last 85.93) - open/close 59337 (time 1761288596.83 total 677.48 last 80.78) - open/close 60000 (time 1761288605.10 total 685.75 last 80.16) - open/close 60773 (time 1761288615.10 total 695.75 last 77.27) - open/close 61553 (time 1761288625.11 total 705.76 last 77.94) - open/close 62361 (time 1761288635.11 total 715.76 last 80.79) - open/close 63175 (time 1761288645.13 total 725.78 last 81.29) - open/close 64011 (time 1761288655.14 total 735.79 last 83.52) total: 64768 open/close in 745.25 seconds: 86.91 ops/second - /unlink 750 (time 1761288678.02 total 10.01 last 74.90) - /unlink 1555 (time 1761288688.02 total 20.01 last 80.50) - /unlink 2390 (time 1761288698.03 total 30.02 last 83.45) - /unlink 3255 (time 1761288708.04 total 40.04 last 86.37) - /unlink 4159 (time 1761288718.05 total 50.04 last 90.35) - /unlink 5102 (time 1761288728.05 total 60.05 last 94.25) - /unlink 6197 (time 1761288738.06 total 70.05 last 109.42) - /unlink 7290 (time 1761288748.06 total 80.06 last 109.26) - /unlink 8332 (time 1761288758.06 total 90.06 last 104.17) - /unlink 9842 (time 1761288768.07 total 100.06 last 150.91) - /unlink 11244 (time 1761288778.07 total 110.07 last 140.17) - /unlink 12827 (time 1761288788.08 total 120.07 last 158.21) - /unlink 14079 (time 1761288798.08 total 130.08 last 125.13) - /unlink 15618 (time 1761288808.09 total 140.08 last 153.87) - /unlink 17130 (time 1761288818.09 total 150.09 last 151.09) - /unlink 18529 (time 1761288828.09 total 160.09 last 139.89) - /unlink 20000 (time 1761288838.04 total 170.03 last 147.93) - /unlink 21423 (time 1761288848.04 total 180.04 last 142.21) - /unlink 22780 (time 1761288858.05 total 190.04 last 135.67) - /unlink 24234 (time 1761288868.05 total 200.04 last 145.40) - /unlink 25479 (time 1761288878.05 total 210.05 last 124.41) - /unlink 26750 (time 1761288888.06 total 220.06 last 127.00) - /unlink 28131 (time 1761288898.06 total 230.06 last 138.09) - /unlink 29613 (time 1761288908.06 total 240.06 last 148.18) - /unlink 30000 (time 1761288910.48 total 242.48 last 160.05) - /unlink 31425 (time 1761288920.49 total 252.48 last 142.39) - /unlink 33144 (time 1761288930.49 total 262.49 last 171.83) - /unlink 34767 (time 1761288940.50 total 272.49 last 162.18) - /unlink 36169 (time 1761288950.50 total 282.50 last 140.16) - /unlink 37599 (time 1761288960.51 total 292.50 last 142.92) - /unlink 38874 (time 1761288970.51 total 302.50 last 127.48) - /unlink 40000 (time 1761288978.84 total 310.83 last 135.21) - /unlink 41251 (time 1761288988.84 total 320.84 last 125.03) - /unlink 42397 (time 1761288998.85 total 330.84 last 114.56) - /unlink 43356 (time 1761289008.86 total 340.85 last 95.77) - /unlink 44395 (time 1761289018.87 total 350.87 last 103.78) - /unlink 46135 (time 1761289028.87 total 360.87 last 173.98) - /unlink 47869 (time 1761289038.87 total 370.87 last 173.37) - /unlink 49900 (time 1761289048.88 total 380.87 last 202.98) - /unlink 52328 (time 1761289058.89 total 390.88 last 242.62) - /unlink 54195 (time 1761289068.89 total 400.88 last 186.66) - /unlink 55940 (time 1761289078.90 total 410.89 last 174.36) - /unlink 57517 (time 1761289088.90 total 420.89 last 157.68) - /unlink 59325 (time 1761289098.90 total 430.90 last 180.73) - /unlink 60000 (time 1761289102.35 total 434.34 last 195.96) - /unlink 61916 (time 1761289112.35 total 444.34 last 191.59) - /unlink 63782 (time 1761289122.35 total 454.35 last 186.53) total: 64768 /unlink in 458.30 seconds: 141.32 ops/second Waiting for MDT destroys to complete oleg121-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 - open/close 1479 (time 1761289145.70 total 10.01 last 147.82) - open/close 2687 (time 1761289155.71 total 20.01 last 120.77) - open/close 3899 (time 1761289165.71 total 30.01 last 121.17) - open/close 5265 (time 1761289175.71 total 40.01 last 136.60) - open/close 6702 (time 1761289185.71 total 50.02 last 143.64) - open/close 8162 (time 1761289195.72 total 60.02 last 145.88) - open/close 9644 (time 1761289205.72 total 70.02 last 148.18) - open/close 11337 (time 1761289215.72 total 80.03 last 169.28) - open/close 12673 (time 1761289225.73 total 90.03 last 133.49) - open/close 13976 (time 1761289235.74 total 100.04 last 130.24) - open/close 15214 (time 1761289245.74 total 110.04 last 123.76) - open/close 16820 (time 1761289255.74 total 120.04 last 160.56) - open/close 18300 (time 1761289265.74 total 130.05 last 147.97) - open/close 19888 (time 1761289275.75 total 140.05 last 158.75) - open/close 21651 (time 1761289285.75 total 150.05 last 176.27) - open/close 23297 (time 1761289295.75 total 160.05 last 164.55) - open/close 24936 (time 1761289305.75 total 170.05 last 163.89) - open/close 26642 (time 1761289315.75 total 180.06 last 170.54) - open/close 28321 (time 1761289325.76 total 190.06 last 167.83) - open/close 29728 (time 1761289335.76 total 200.07 last 140.65) - open/close 30000 (time 1761289337.90 total 202.20 last 127.26) - open/close 31288 (time 1761289347.90 total 212.21 last 128.75) - open/close 32539 (time 1761289357.91 total 222.21 last 125.07) - open/close 33925 (time 1761289367.91 total 232.21 last 138.53) - open/close 35635 (time 1761289377.91 total 242.21 last 171.00) - open/close 37447 (time 1761289387.91 total 252.22 last 181.18) - open/close 38975 (time 1761289397.91 total 262.22 last 152.77) - open/close 40000 (time 1761289404.05 total 268.35 last 167.19) - open/close 41575 (time 1761289414.05 total 278.35 last 157.48) - open/close 43384 (time 1761289424.06 total 288.36 last 180.71) - open/close 45064 (time 1761289434.06 total 298.37 last 167.91) - open/close 46725 (time 1761289444.06 total 308.37 last 166.07) - open/close 48398 (time 1761289454.07 total 318.37 last 167.27) - open/close 50000 (time 1761289464.01 total 328.31 last 161.14) - open/close 51786 (time 1761289474.01 total 338.32 last 178.53) - open/close 53474 (time 1761289484.01 total 348.32 last 168.79) - open/close 55376 (time 1761289494.02 total 358.32 last 190.14) - open/close 57419 (time 1761289504.02 total 368.32 last 204.26) - open/close 59405 (time 1761289514.02 total 378.32 last 198.53) - open/close 60000 (time 1761289517.16 total 381.46 last 189.59) - open/close 61976 (time 1761289527.16 total 391.47 last 197.55) - open/close 64295 (time 1761289537.17 total 401.47 last 231.82) total: 64768 open/close in 403.52 seconds: 160.51 ops/second - /unlink 3416 (time 1761289550.91 total 10.00 last 341.56) - /unlink 6897 (time 1761289560.92 total 20.00 last 348.03) - /unlink 10000 (time 1761289570.32 total 29.41 last 329.85) - /unlink 12899 (time 1761289580.32 total 39.41 last 289.86) - /unlink 15911 (time 1761289590.33 total 49.41 last 301.11) - /unlink 20000 (time 1761289599.45 total 58.54 last 448.02) - /unlink 24677 (time 1761289609.46 total 68.54 last 467.63) - /unlink 29495 (time 1761289619.46 total 78.54 last 481.73) - /unlink 34620 (time 1761289629.46 total 88.54 last 512.48) - /unlink 39668 (time 1761289639.46 total 98.55 last 504.74) - /unlink 44534 (time 1761289649.46 total 108.55 last 486.55) - /unlink 49188 (time 1761289659.46 total 118.55 last 465.30) - /unlink 50000 (time 1761289661.74 total 120.82 last 357.08) - /unlink 54114 (time 1761289671.74 total 130.83 last 411.26) - /unlink 58273 (time 1761289681.74 total 140.83 last 415.90) - /unlink 60000 (time 1761289685.19 total 144.28 last 499.76) total: 64768 /unlink in 152.27 seconds: 425.34 ops/second Waiting for MDT destroys to complete oleg121-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 - open/close 2341 (time 1761289713.67 total 10.00 last 233.99) - open/close 4470 (time 1761289723.67 total 20.01 last 212.83) - open/close 6585 (time 1761289733.67 total 30.01 last 211.45) - open/close 8387 (time 1761289743.68 total 40.02 last 180.07) - open/close 9281 (time 1761289753.68 total 50.02 last 89.37) - open/close 10000 (time 1761289760.11 total 56.45 last 111.82) - open/close 10913 (time 1761289770.12 total 66.46 last 91.25) - open/close 11669 (time 1761289780.13 total 76.47 last 75.52) - open/close 12470 (time 1761289790.13 total 86.47 last 80.06) - open/close 13283 (time 1761289800.14 total 96.48 last 81.27) - open/close 14203 (time 1761289810.14 total 106.48 last 91.96) - open/close 15057 (time 1761289820.14 total 116.48 last 85.38) - open/close 15953 (time 1761289830.16 total 126.49 last 89.49) - open/close 16911 (time 1761289840.16 total 136.50 last 95.79) - open/close 17960 (time 1761289850.17 total 146.51 last 104.78) - open/close 19004 (time 1761289860.18 total 156.52 last 104.29) - open/close 19882 (time 1761289870.18 total 166.52 last 87.78) - open/close 20782 (time 1761289880.18 total 176.52 last 89.98) - open/close 21674 (time 1761289890.19 total 186.53 last 89.14) - open/close 22615 (time 1761289900.19 total 196.53 last 94.09) - open/close 23529 (time 1761289910.20 total 206.53 last 91.37) - open/close 24513 (time 1761289920.20 total 216.54 last 98.38) - open/close 25593 (time 1761289930.20 total 226.54 last 108.00) - open/close 26682 (time 1761289940.20 total 236.54 last 108.84) - open/close 27759 (time 1761289950.21 total 246.54 last 107.70) - open/close 28721 (time 1761289960.21 total 256.55 last 96.16) - open/close 29810 (time 1761289970.21 total 266.55 last 108.89) - open/close 30836 (time 1761289980.22 total 276.55 last 102.53) - open/close 31826 (time 1761289990.23 total 286.56 last 98.92) - open/close 32798 (time 1761290000.23 total 296.57 last 97.17) - open/close 33790 (time 1761290010.23 total 306.57 last 99.16) - open/close 34734 (time 1761290020.23 total 316.57 last 94.37) - open/close 35603 (time 1761290030.24 total 326.58 last 86.84) - open/close 36480 (time 1761290040.25 total 336.59 last 87.60) - open/close 37395 (time 1761290050.26 total 346.60 last 91.39) - open/close 38313 (time 1761290060.27 total 356.61 last 91.76) - open/close 39184 (time 1761290070.28 total 366.62 last 87.02) - open/close 40000 (time 1761290078.87 total 375.20 last 95.01) - open/close 40938 (time 1761290088.88 total 385.21 last 93.71) - open/close 41916 (time 1761290098.88 total 395.22 last 97.78) - open/close 42917 (time 1761290108.88 total 405.22 last 100.08) - open/close 43955 (time 1761290118.88 total 415.22 last 103.76) - open/close 44876 (time 1761290128.89 total 425.23 last 92.05) - open/close 45844 (time 1761290138.89 total 435.23 last 96.78) - open/close 46739 (time 1761290148.90 total 445.24 last 89.42) - open/close 47717 (time 1761290158.91 total 455.25 last 97.70) - open/close 48666 (time 1761290168.91 total 465.25 last 94.87) - open/close 49598 (time 1761290178.92 total 475.26 last 93.10) - open/close 50000 (time 1761290183.41 total 479.75 last 89.55) - open/close 50943 (time 1761290193.42 total 489.76 last 94.22) - open/close 51914 (time 1761290203.42 total 499.76 last 97.06) - open/close 52842 (time 1761290213.43 total 509.77 last 92.77) - open/close 53864 (time 1761290223.43 total 519.77 last 102.19) - open/close 54802 (time 1761290233.43 total 529.77 last 93.77) - open/close 55718 (time 1761290243.44 total 539.78 last 91.55) - open/close 56608 (time 1761290253.45 total 549.79 last 88.89) - open/close 57496 (time 1761290263.45 total 559.79 last 88.78) - open/close 58466 (time 1761290273.45 total 569.79 last 96.98) - open/close 59380 (time 1761290283.46 total 579.80 last 91.34) - open/close 60000 (time 1761290289.83 total 586.17 last 97.34) - open/close 60936 (time 1761290299.84 total 596.18 last 93.53) - open/close 61856 (time 1761290309.85 total 606.19 last 91.86) - open/close 62754 (time 1761290319.85 total 616.19 last 89.79) - open/close 63631 (time 1761290329.87 total 626.21 last 87.56) - open/close 64555 (time 1761290339.87 total 636.21 last 92.38) total: 64768 open/close in 638.69 seconds: 101.41 ops/second - /unlink 883 (time 1761290355.08 total 10.00 last 88.29) - /unlink 1767 (time 1761290365.08 total 20.00 last 88.39) - /unlink 2718 (time 1761290375.10 total 30.02 last 94.96) - /unlink 3726 (time 1761290385.10 total 40.02 last 100.75) - /unlink 4768 (time 1761290395.11 total 50.03 last 104.14) - /unlink 5812 (time 1761290405.11 total 60.03 last 104.38) - /unlink 6864 (time 1761290415.12 total 70.04 last 105.12) - /unlink 7895 (time 1761290425.12 total 80.04 last 103.10) - /unlink 8798 (time 1761290435.12 total 90.04 last 90.28) - /unlink 9900 (time 1761290445.13 total 100.05 last 110.09) - /unlink 10975 (time 1761290455.14 total 110.05 last 107.44) - /unlink 11994 (time 1761290465.14 total 120.06 last 101.83) - /unlink 12984 (time 1761290475.16 total 130.07 last 98.87) - /unlink 13993 (time 1761290485.16 total 140.08 last 100.83) - /unlink 15012 (time 1761290495.17 total 150.08 last 101.87) - /unlink 16184 (time 1761290505.17 total 160.09 last 117.16) - /unlink 17196 (time 1761290515.17 total 170.09 last 101.17) - /unlink 18202 (time 1761290525.17 total 180.09 last 100.58) - /unlink 19113 (time 1761290535.17 total 190.09 last 91.09) - /unlink 20000 (time 1761290544.56 total 199.48 last 94.46) - /unlink 21082 (time 1761290554.57 total 209.49 last 108.15) - /unlink 22013 (time 1761290564.57 total 219.49 last 93.10) - /unlink 22977 (time 1761290574.57 total 229.49 last 96.35) - /unlink 23955 (time 1761290584.58 total 239.50 last 97.72) - /unlink 25003 (time 1761290594.59 total 249.50 last 104.77) - /unlink 26050 (time 1761290604.59 total 259.51 last 104.66) - /unlink 27277 (time 1761290614.59 total 269.51 last 122.68) - /unlink 28681 (time 1761290624.60 total 279.51 last 140.33) - /unlink 30000 (time 1761290634.54 total 289.45 last 132.70) - /unlink 31495 (time 1761290644.54 total 299.46 last 149.40) - /unlink 32675 (time 1761290654.55 total 309.46 last 117.96) - /unlink 34014 (time 1761290664.55 total 319.46 last 133.90) - /unlink 35542 (time 1761290674.55 total 329.47 last 152.72) - /unlink 36935 (time 1761290684.55 total 339.47 last 139.27) - /unlink 38312 (time 1761290694.56 total 349.48 last 137.59) - /unlink 39593 (time 1761290704.56 total 359.48 last 128.06) - /unlink 40000 (time 1761290707.87 total 362.79 last 123.14) - /unlink 41322 (time 1761290717.87 total 372.79 last 132.18) - /unlink 42565 (time 1761290727.88 total 382.80 last 124.22) - /unlink 44178 (time 1761290737.88 total 392.80 last 161.22) - /unlink 45611 (time 1761290747.89 total 402.80 last 143.24) - /unlink 47037 (time 1761290757.89 total 412.81 last 142.57) - /unlink 48391 (time 1761290767.89 total 422.81 last 135.35) - /unlink 50000 (time 1761290775.07 total 429.99 last 224.19) - /unlink 52715 (time 1761290785.07 total 439.99 last 271.45) - /unlink 55626 (time 1761290795.07 total 449.99 last 291.09) - /unlink 58637 (time 1761290805.07 total 459.99 last 301.00) - /unlink 60000 (time 1761290809.80 total 464.72 last 288.43) - /unlink 62860 (time 1761290819.80 total 474.72 last 285.94) total: 64768 /unlink in 482.33 seconds: 134.28 ops/second Waiting for MDT destroys to complete oleg121-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg121-client: oleg121-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg121-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg121-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg121-server stop mds service on oleg121-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg121-server stop mds service on oleg121-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg121-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing unload_modules_local oleg121-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg121-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg121-client: oleg121-client: ssh exited with exit code 2 pdsh@oleg121-client: oleg121-server: ssh exited with exit code 2 pdsh@oleg121-client: oleg121-client: ssh exited with exit code 2 pdsh@oleg121-client: oleg121-server: ssh exited with exit code 2 PASS 106 (6204s) Stopping clients: oleg121-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg121-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg121-client: oleg121-server: ssh exited with exit code 2 oleg121-server: oleg121-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg121-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 6392 sec ========== 03:28:27 (1761290907) === conf-sanity: start cleanup 03:28:28 (1761290908) === === conf-sanity: finish cleanup 03:28:30 (1761290910) ===