== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 13:54:40 (1772304880) Stopping clients: oleg355-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg355-client.virtnet /mnt/lustre2 (opts:-f) oleg355-server: oleg355-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg355-server' oleg355-server: oleg355-server.virtnet: executing load_modules_local oleg355-server: Loading modules from /home/green/git/lustre-release/lustre oleg355-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg355-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg355-server: detected 4 online CPUs by sysfs oleg355-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg355-server' oleg355-server: oleg355-server.virtnet: executing load_modules_local oleg355-server: Loading modules from /home/green/git/lustre-release/lustre oleg355-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg355-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg355-server: detected 4 online CPUs by sysfs oleg355-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg355-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg355-server: oleg355-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg355-client: oleg355-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg355-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg355-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg355-server: oleg355-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg355-client: oleg355-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg355-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg355-client.virtnet: -o user_xattr,flock oleg355-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 4390 (time 1772304929.68 total 10.00 last 438.86) - open/close 8301 (time 1772304939.68 total 20.00 last 391.06) total: 10000 open/close in 24.19 seconds: 413.37 ops/second - unlinked 0 (time 1772304945 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4575 (time 1772304971.08 total 10.00 last 457.39) - open/close 9286 (time 1772304981.08 total 20.00 last 471.03) total: 10000 open/close in 21.32 seconds: 469.03 ops/second - unlinked 0 (time 1772304983 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4376 (time 1772305009.20 total 10.00 last 437.58) - open/close 8468 (time 1772305019.20 total 20.00 last 409.15) total: 10000 open/close in 23.32 seconds: 428.73 ops/second - unlinked 0 (time 1772305023 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4449 (time 1772305048.81 total 10.00 last 444.89) - open/close 8766 (time 1772305058.81 total 20.00 last 431.62) total: 10000 open/close in 23.01 seconds: 434.56 ops/second - unlinked 0 (time 1772305063 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4038 (time 1772305088.64 total 10.00 last 403.77) - open/close 8099 (time 1772305098.65 total 20.00 last 405.99) total: 10000 open/close in 25.33 seconds: 394.84 ops/second - unlinked 0 (time 1772305105 ; total 0 ; last 0) total: 10000 unlinks in 17 seconds: 588.235291 unlinks/second - open/close 3405 (time 1772305134.36 total 10.00 last 340.49) - open/close 6717 (time 1772305144.37 total 20.00 last 331.07) total: 10000 open/close in 29.74 seconds: 336.30 ops/second - unlinked 0 (time 1772305155 ; total 0 ; last 0) total: 10000 unlinks in 16 seconds: 625.000000 unlinks/second - open/close 3835 (time 1772305182.55 total 10.00 last 383.46) - open/close 7581 (time 1772305192.56 total 20.00 last 374.51) total: 10000 open/close in 26.38 seconds: 379.10 ops/second - unlinked 0 (time 1772305200 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3341 (time 1772305226.91 total 10.00 last 334.03) - open/close 7090 (time 1772305236.91 total 20.00 last 374.80) - open/close 9507 (time 1772305246.92 total 30.01 last 241.67) total: 10000 open/close in 31.64 seconds: 316.03 ops/second - unlinked 0 (time 1772305250 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3193 (time 1772305277.11 total 10.00 last 319.14) - open/close 6013 (time 1772305287.12 total 20.01 last 281.88) - open/close 8759 (time 1772305297.12 total 30.01 last 274.57) total: 10000 open/close in 33.78 seconds: 296.00 ops/second - unlinked 0 (time 1772305302 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 2883 (time 1772305332.14 total 10.00 last 288.21) - open/close 6364 (time 1772305342.14 total 20.00 last 348.07) - open/close 9200 (time 1772305352.14 total 30.00 last 283.59) total: 10000 open/close in 32.55 seconds: 307.19 ops/second - unlinked 0 (time 1772305356 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 3708 (time 1772305386.37 total 10.00 last 370.71) - open/close 6947 (time 1772305396.37 total 20.00 last 323.87) total: 10000 open/close in 29.45 seconds: 339.57 ops/second - unlinked 0 (time 1772305407 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5504 376064 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210288 s, 5.0 MB/s fail_loc=0x80000204 Failing ost1 on oleg355-server Stopping /mnt/lustre-ost1 (opts:) on oleg355-server reboot facets: ost1 Failover ost1 to oleg355-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg355-server: oleg355-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg355-client: oleg355-server: ssh exited with exit code 1 Started lustre-OST0000 oleg355-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1772305457.20 total 4.40 last 0.00) total: 100 open/close in 4.96 seconds: 20.15 ops/second umount lustre on /mnt/lustre..... Stopping client oleg355-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg355-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg355-server stop mds service on oleg355-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg355-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg355-server' oleg355-server: oleg355-server.virtnet: executing unload_modules_local oleg355-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg355-server: LNET ready to unload oleg355-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded.