== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 10:26:40 (1760452000) Stopping clients: oleg433-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg433-client.virtnet /mnt/lustre2 (opts:-f) oleg433-server: oleg433-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing load_modules_local oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg433-server: Loading modules from /home/green/git/lustre-release/lustre oleg433-server: detected 4 online CPUs by sysfs oleg433-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing load_modules_local oleg433-server: Loading modules from /home/green/git/lustre-release/lustre oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg433-server: detected 4 online CPUs by sysfs oleg433-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg433-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg433-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg433-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg433-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg433-client.virtnet: -o user_xattr,flock oleg433-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 1695 (time 1760452060.46 total 10.00 last 169.42) - open/close 4030 (time 1760452070.46 total 20.00 last 233.50) - open/close 6531 (time 1760452080.46 total 30.01 last 250.08) - open/close 9291 (time 1760452090.47 total 40.01 last 275.77) total: 10000 open/close in 43.06 seconds: 232.22 ops/second - unlinked 0 (time 1760452095 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 2391 (time 1760452133.75 total 10.00 last 239.04) - open/close 4717 (time 1760452143.75 total 20.00 last 232.57) - open/close 6809 (time 1760452153.76 total 30.01 last 209.17) - open/close 8574 (time 1760452163.76 total 40.01 last 176.47) total: 10000 open/close in 48.75 seconds: 205.13 ops/second - unlinked 0 (time 1760452174 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 2757 (time 1760452213.07 total 10.00 last 275.68) - open/close 5165 (time 1760452223.08 total 20.01 last 240.67) - open/close 7346 (time 1760452233.08 total 30.01 last 218.04) - open/close 9132 (time 1760452243.09 total 40.01 last 178.53) total: 10000 open/close in 44.41 seconds: 225.18 ops/second - unlinked 0 (time 1760452249 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 2608 (time 1760452287.35 total 10.00 last 260.79) - open/close 4765 (time 1760452297.35 total 20.00 last 215.68) - open/close 6569 (time 1760452307.35 total 30.00 last 180.39) - open/close 8527 (time 1760452317.35 total 40.00 last 195.76) total: 10000 open/close in 47.31 seconds: 211.37 ops/second - unlinked 0 (time 1760452326 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 1892 (time 1760452365.81 total 10.00 last 189.18) - open/close 3538 (time 1760452375.81 total 20.01 last 164.53) - open/close 5483 (time 1760452385.81 total 30.01 last 194.43) - open/close 7502 (time 1760452395.82 total 40.01 last 201.84) - open/close 9588 (time 1760452405.82 total 50.01 last 208.58) total: 10000 open/close in 51.83 seconds: 192.93 ops/second - unlinked 0 (time 1760452409 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 1863 (time 1760452447.06 total 10.00 last 186.23) - open/close 3833 (time 1760452457.06 total 20.01 last 196.97) - open/close 5858 (time 1760452467.07 total 30.01 last 202.46) - open/close 7802 (time 1760452477.07 total 40.01 last 194.31) - open/close 9837 (time 1760452487.07 total 50.02 last 203.44) total: 10000 open/close in 50.83 seconds: 196.75 ops/second - unlinked 0 (time 1760452489 ; total 0 ; last 0) total: 10000 unlinks in 27 seconds: 370.370361 unlinks/second - open/close 2192 (time 1760452528.05 total 10.00 last 219.20) - open/close 4213 (time 1760452538.05 total 20.00 last 202.10) - open/close 6847 (time 1760452548.06 total 30.01 last 263.24) - open/close 9571 (time 1760452558.06 total 40.01 last 272.35) total: 10000 open/close in 41.51 seconds: 240.89 ops/second - unlinked 0 (time 1760452561 ; total 0 ; last 0) total: 10000 unlinks in 24 seconds: 416.666656 unlinks/second - open/close 2431 (time 1760452597.30 total 10.00 last 243.08) - open/close 4847 (time 1760452607.31 total 20.00 last 241.55) - open/close 7405 (time 1760452617.31 total 30.00 last 255.75) - open/close 9600 (time 1760452627.31 total 40.01 last 219.46) total: 10000 open/close in 42.03 seconds: 237.92 ops/second - unlinked 0 (time 1760452631 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2246 (time 1760452664.44 total 10.00 last 224.51) - open/close 4415 (time 1760452674.44 total 20.01 last 216.84) - open/close 6606 (time 1760452684.44 total 30.01 last 219.10) - open/close 8711 (time 1760452694.44 total 40.01 last 210.43) total: 10000 open/close in 46.76 seconds: 213.84 ops/second - unlinked 0 (time 1760452703 ; total 0 ; last 0) total: 10000 unlinks in 24 seconds: 416.666656 unlinks/second - open/close 2251 (time 1760452739.67 total 10.00 last 225.08) - open/close 5157 (time 1760452749.67 total 20.00 last 290.52) - open/close 8195 (time 1760452759.68 total 30.01 last 303.73) total: 10000 open/close in 36.40 seconds: 274.76 ops/second - unlinked 0 (time 1760452767 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 2090 (time 1760452801.16 total 10.00 last 208.96) - open/close 4598 (time 1760452811.17 total 20.01 last 250.47) - open/close 7124 (time 1760452821.17 total 30.02 last 252.60) - open/close 9566 (time 1760452831.18 total 40.02 last 244.11) total: 10000 open/close in 41.82 seconds: 239.12 ops/second - unlinked 0 (time 1760452834 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5376 376192 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 3072 377856 1% /mnt/lustre[OST:0] filesystem_summary: 382976 3072 377856 1% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190071 s, 5.5 MB/s fail_loc=0x80000204 Failing ost1 on oleg433-server Stopping /mnt/lustre-ost1 (opts:) on oleg433-server reboot facets: ost1 Failover ost1 to oleg433-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Started lustre-OST0000 oleg433-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec total: 100 open/close in 2.66 seconds: 37.60 ops/second umount lustre on /mnt/lustre..... Stopping client oleg433-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg433-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg433-server stop mds service on oleg433-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg433-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing unload_modules_local oleg433-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg433-server: LNET ready to unload oleg433-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded.