== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 21:40:23 (1767840023) Stopping clients: oleg340-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:-f) oleg340-server: oleg340-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg340-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg340-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg340-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 2034 (time 1767840084.97 total 10.00 last 203.33) - open/close 4020 (time 1767840094.97 total 20.00 last 198.60) - open/close 5983 (time 1767840104.98 total 30.01 last 196.11) - open/close 8242 (time 1767840114.98 total 40.02 last 225.84) total: 10000 open/close in 46.25 seconds: 216.21 ops/second - unlinked 0 (time 1767840123 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1297 (time 1767840171.43 total 10.01 last 129.61) - open/close 2706 (time 1767840181.44 total 20.01 last 140.81) - open/close 4125 (time 1767840191.44 total 30.02 last 141.88) - open/close 6055 (time 1767840201.44 total 40.02 last 192.94) - open/close 7908 (time 1767840211.45 total 50.02 last 185.20) - open/close 9934 (time 1767840221.45 total 60.03 last 202.55) total: 10000 open/close in 60.33 seconds: 165.77 ops/second - unlinked 0 (time 1767840223 ; total 0 ; last 0) total: 10000 unlinks in 32 seconds: 312.500000 unlinks/second - open/close 1971 (time 1767840267.39 total 10.00 last 197.07) - open/close 4826 (time 1767840277.39 total 20.00 last 285.43) - open/close 7386 (time 1767840287.40 total 30.01 last 255.87) - open/close 9922 (time 1767840297.40 total 40.01 last 253.58) total: 10000 open/close in 40.41 seconds: 247.47 ops/second - unlinked 0 (time 1767840299 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 1795 (time 1767840332.96 total 10.00 last 179.45) - open/close 3588 (time 1767840342.97 total 20.01 last 179.25) - open/close 5641 (time 1767840352.97 total 30.01 last 205.18) - open/close 7783 (time 1767840362.97 total 40.01 last 214.19) - open/close 9532 (time 1767840372.98 total 50.02 last 174.83) total: 10000 open/close in 52.43 seconds: 190.73 ops/second - unlinked 0 (time 1767840377 ; total 0 ; last 0) total: 10000 unlinks in 17 seconds: 588.235291 unlinks/second - open/close 4134 (time 1767840406.44 total 10.00 last 413.40) - open/close 8139 (time 1767840416.44 total 20.00 last 400.46) total: 10000 open/close in 24.46 seconds: 408.83 ops/second - unlinked 0 (time 1767840422 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4954 (time 1767840447.78 total 10.00 last 495.32) - open/close 9331 (time 1767840457.79 total 20.00 last 437.66) total: 10000 open/close in 21.52 seconds: 464.76 ops/second - unlinked 0 (time 1767840460 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4998 (time 1767840484.85 total 10.00 last 499.70) total: 10000 open/close in 19.92 seconds: 502.09 ops/second - unlinked 0 (time 1767840496 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4731 (time 1767840518.77 total 10.00 last 473.09) - open/close 9495 (time 1767840528.77 total 20.00 last 476.32) total: 10000 open/close in 20.93 seconds: 477.81 ops/second - unlinked 0 (time 1767840530 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4783 (time 1767840554.89 total 10.00 last 478.21) - open/close 9125 (time 1767840564.89 total 20.00 last 434.16) total: 10000 open/close in 21.77 seconds: 459.30 ops/second - unlinked 0 (time 1767840568 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 4728 (time 1767840592.08 total 10.00 last 472.77) - open/close 9041 (time 1767840602.08 total 20.00 last 431.26) total: 10000 open/close in 22.39 seconds: 446.64 ops/second - unlinked 0 (time 1767840605 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4913 (time 1767840630.49 total 10.00 last 491.30) - open/close 9622 (time 1767840640.50 total 20.00 last 470.80) total: 10000 open/close in 21.02 seconds: 475.83 ops/second - unlinked 0 (time 1767840642 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5632 375936 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229833 s, 4.6 MB/s fail_loc=0x80000204 Failing ost1 on oleg340-server Stopping /mnt/lustre-ost1 (opts:) on oleg340-server reboot facets: ost1 Failover ost1 to oleg340-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1767840688.19 total 4.71 last 0.00) total: 100 open/close in 5.27 seconds: 18.99 ops/second umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing unload_modules_local oleg340-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg340-server: LNET ready to unload oleg340-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded.