== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 09:13:27 (1759842807) Stopping clients: oleg459-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg459-client.virtnet /mnt/lustre2 (opts:-f) oleg459-server: oleg459-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg459-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg459-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg459-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg459-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg459-client.virtnet: -o user_xattr,flock oleg459-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 4730 (time 1759842856.82 total 10.00 last 472.99) - open/close 9456 (time 1759842866.82 total 20.00 last 472.50) total: 10000 open/close in 21.13 seconds: 473.29 ops/second - unlinked 0 (time 1759842869 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4580 (time 1759842893.69 total 10.00 last 457.93) - open/close 9040 (time 1759842903.69 total 20.00 last 445.97) total: 10000 open/close in 22.09 seconds: 452.75 ops/second - unlinked 0 (time 1759842907 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 4452 (time 1759842930.96 total 10.00 last 445.19) - open/close 9035 (time 1759842940.96 total 20.00 last 458.23) total: 10000 open/close in 22.13 seconds: 451.89 ops/second - unlinked 0 (time 1759842944 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4499 (time 1759842968.93 total 10.00 last 449.90) - open/close 8992 (time 1759842978.93 total 20.00 last 449.30) total: 10000 open/close in 22.26 seconds: 449.15 ops/second - unlinked 0 (time 1759842982 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4625 (time 1759843006.97 total 10.00 last 462.45) - open/close 9292 (time 1759843016.97 total 20.00 last 466.62) total: 10000 open/close in 21.63 seconds: 462.27 ops/second - unlinked 0 (time 1759843020 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4599 (time 1759843044.50 total 10.00 last 459.86) - open/close 9228 (time 1759843054.50 total 20.00 last 462.80) total: 10000 open/close in 21.76 seconds: 459.53 ops/second - unlinked 0 (time 1759843057 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 5159 (time 1759843080.71 total 10.00 last 515.83) - open/close 9936 (time 1759843090.71 total 20.01 last 477.47) total: 10000 open/close in 20.23 seconds: 494.21 ops/second - unlinked 0 (time 1759843092 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4784 (time 1759843114.42 total 10.00 last 478.35) - open/close 9138 (time 1759843124.42 total 20.00 last 435.38) total: 10000 open/close in 21.99 seconds: 454.71 ops/second - unlinked 0 (time 1759843127 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4605 (time 1759843152.68 total 10.00 last 460.45) - open/close 9234 (time 1759843162.68 total 20.00 last 462.81) total: 10000 open/close in 21.60 seconds: 462.88 ops/second - unlinked 0 (time 1759843165 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4819 (time 1759843189.71 total 10.00 last 481.84) - open/close 9896 (time 1759843199.71 total 20.00 last 507.61) total: 10000 open/close in 20.19 seconds: 495.28 ops/second - unlinked 0 (time 1759843201 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4840 (time 1759843223.90 total 10.00 last 483.79) - open/close 9958 (time 1759843233.90 total 20.01 last 511.74) total: 10000 open/close in 20.09 seconds: 497.71 ops/second - unlinked 0 (time 1759843235 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5760 375808 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180747 s, 5.8 MB/s fail_loc=0x80000204 Failing ost1 on oleg459-server Stopping /mnt/lustre-ost1 (opts:) on oleg459-server reboot facets: ost1 Failover ost1 to oleg459-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Started lustre-OST0000 oleg459-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1759843276.21 total 3.66 last 0.00) total: 100 open/close in 4.25 seconds: 23.55 ops/second umount lustre on /mnt/lustre..... Stopping client oleg459-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg459-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg459-server stop mds service on oleg459-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg459-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing unload_modules_local oleg459-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg459-server: LNET ready to unload oleg459-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded.