-----============= acceptance-small: conf-sanity ============----- Wed Mar 6 23:39:46 EST 2024 excepting tests: 32newtarball 110 skipping tests SLOW=no: 45 69 106 111 114 Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg154-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg154-server oleg154-server: oleg154-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg154-server: oleg154-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg154-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 35b: Continue reconnection retries, if the active server is busy ========================================================== 23:40:46 (1709786446) start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre debug=ha conf-sanity.sh test_35b 2024-03-0623h41m01s Set up a fake failnode for the MDS at_max=0 at_max=0 Injecting EBUSY on MDS fail_loc=0x80000136 mdc.lustre-MDT0000-mdc-ffff8800aa38f000.stats=clear Creating a test file and stat it File: '/mnt/lustre/d35b.conf-sanity/f35b.conf-sanity' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934274 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-06 23:41:44.000000000 -0500 Modify: 2024-03-06 23:41:44.000000000 -0500 Change: 2024-03-06 23:41:44.000000000 -0500 Birth: - Stop injecting EBUSY on MDS fail_loc=0 done at_max=600 at_max=600 Debug log: 54 lines, 54 kept, 0 dropped, 0 bad. umount lustre on /mnt/lustre..... Stopping client oleg154-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server unloading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing unload_modules_local modules unloaded. oleg154-server: oleg154-server: tunefs.lustre FATAL: Device lustre-mdt1/mdt1 has not been formatted with mkfs.lustre oleg154-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg154-server: oleg154-server: tunefs.lustre FATAL: Device lustre-ost1/ost1 has not been formatted with mkfs.lustre oleg154-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg154-server: oleg154-server: tunefs.lustre FATAL: Device lustre-ost2/ost2 has not been formatted with mkfs.lustre oleg154-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 19 checking for existing Lustre data: not found tunefs failed, reformatting instead Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 oleg154-server: oleg154-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions oleg154-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg154-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg154-server: oleg154-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg154-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server PASS 35b (125s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 23:42:53 (1709786573) start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre copying 10 files to /mnt/lustre/d38.conf-sanity tar: Removing leading `/' from member names umount lustre on /mnt/lustre..... Stopping client oleg154-client.virtnet /mnt/lustre (opts:) osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=44 stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server delete lov_objid file on MDS 000000 43 000008 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=2 umount lustre on /mnt/lustre..... Stopping client oleg154-client.virtnet /mnt/lustre (opts:) stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server 000000 2 000008 oleg154-server: 8+0 records in oleg154-server: 8+0 records out oleg154-server: 4096 bytes (4.1 kB) copied, 0.000615015 s, 6.7 MB/s start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=35 umount lustre on /mnt/lustre..... Stopping client oleg154-client.virtnet /mnt/lustre (opts:) stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server 000000 34 000008 files compared the same umount lustre on /mnt/lustre..... stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server unloading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg154-client: oleg154-client: ssh exited with exit code 2 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 pdsh@oleg154-client: oleg154-client: ssh exited with exit code 2 PASS 38 (74s) error: set_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: setting 'debug_raw_pointers'='0': No such file or directory oleg154-server: error: set_param: param_path 'debug_raw_pointers': No such file or directory oleg154-server: error: set_param: setting 'debug_raw_pointers'='0': No such file or directory error: get_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: setting 'debug_raw_pointers'='Y': No such file or directory oleg154-server: error: get_param: param_path 'debug_raw_pointers': No such file or directory pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 oleg154-server: error: set_param: param_path 'debug_raw_pointers': No such file or directory oleg154-server: error: set_param: setting 'debug_raw_pointers'='Y': No such file or directory == conf-sanity test 51: Verify that mdt_reint handles RMF_MDT_MD correctly when an OST is added ========================================================== 23:44:09 (1709786649) Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 oleg154-server: oleg154-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions oleg154-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg154-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre setup single mount lustre success fail_loc=0x142 start ost2 service on oleg154-server Starting ost2: -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost2/ost2 Started lustre-OST0001 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec stop ost2 service on oleg154-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg154-server umount lustre on /mnt/lustre..... Stopping client oleg154-client.virtnet /mnt/lustre (opts:-f) stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server unloading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing unload_modules_local modules unloaded. oleg154-server: oleg154-server: tunefs.lustre FATAL: Device lustre-mdt1/mdt1 has not been formatted with mkfs.lustre oleg154-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg154-server: oleg154-server: tunefs.lustre FATAL: Device lustre-ost1/ost1 has not been formatted with mkfs.lustre oleg154-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg154-server: oleg154-server: tunefs.lustre FATAL: Device lustre-ost2/ost2 has not been formatted with mkfs.lustre oleg154-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 19 checking for existing Lustre data: not found tunefs failed, reformatting instead Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 oleg154-server: oleg154-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions oleg154-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg154-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg154-server: oleg154-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg154-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server PASS 51 (176s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 23:47:09 (1709786829) start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 98 (time 1709786861.53 total 11.54 last 8.49) - open/close 390 (time 1709786872.29 total 22.31 last 27.12) - open/close 665 (time 1709786882.69 total 32.71 last 26.45) - open/close 820 (time 1709786893.32 total 43.34 last 14.58) - open/close 1040 (time 1709786903.37 total 53.39 last 21.88) - open/close 1213 (time 1709786913.62 total 63.64 last 16.88) - open/close 1433 (time 1709786924.39 total 74.41 last 20.42) - open/close 1664 (time 1709786934.92 total 84.93 last 21.95) - open/close 1912 (time 1709786945.06 total 95.08 last 24.45) - open/close 2106 (time 1709786955.73 total 105.75 last 18.18) - open/close 2370 (time 1709786966.27 total 116.29 last 25.06) - open/close 2606 (time 1709786976.59 total 126.61 last 22.86) - open/close 2872 (time 1709786986.61 total 136.62 last 26.57) - open/close 3167 (time 1709786997.19 total 147.21 last 27.86) - open/close 3409 (time 1709787008.01 total 158.03 last 22.37) - open/close 3663 (time 1709787018.38 total 168.39 last 24.51) - open/close 3853 (time 1709787029.25 total 179.27 last 17.47) - open/close 4125 (time 1709787039.56 total 189.58 last 26.39) - open/close 4485 (time 1709787049.61 total 199.63 last 35.81) - open/close 4690 (time 1709787059.64 total 209.66 last 20.44) - open/close 4861 (time 1709787069.71 total 219.73 last 16.99) - open/close 5098 (time 1709787080.67 total 230.69 last 21.63) - open/close 5225 (time 1709787090.73 total 240.75 last 12.62) - open/close 5402 (time 1709787101.28 total 251.30 last 16.78) - open/close 5645 (time 1709787112.16 total 262.18 last 22.33) - open/close 5813 (time 1709787122.23 total 272.24 last 16.70) - open/close 5957 (time 1709787133.23 total 283.25 last 13.09) - open/close 6070 (time 1709787143.73 total 293.75 last 10.76) - open/close 6380 (time 1709787153.75 total 303.77 last 30.93) - open/close 6433 (time 1709787163.79 total 313.81 last 5.28) - open/close 6578 (time 1709787174.01 total 324.03 last 14.18) - open/close 6741 (time 1709787184.46 total 334.48 last 15.59) - open/close 6919 (time 1709787194.91 total 344.93 last 17.04) - open/close 7041 (time 1709787205.52 total 355.54 last 11.50) - open/close 7189 (time 1709787215.65 total 365.66 last 14.61) - open/close 7306 (time 1709787225.66 total 375.67 last 11.69) - open/close 7435 (time 1709787236.59 total 386.61 last 11.79) - open/close 7620 (time 1709787247.57 total 397.59 last 16.85) - open/close 7843 (time 1709787257.82 total 407.84 last 21.76) - open/close 8123 (time 1709787268.60 total 418.62 last 25.97) - open/close 8260 (time 1709787281.26 total 431.28 last 10.82) - open/close 8365 (time 1709787291.26 total 441.28 last 10.50) - open/close 8682 (time 1709787301.27 total 451.29 last 31.69) - open/close 8848 (time 1709787311.87 total 461.89 last 15.65) - open/close 9004 (time 1709787321.93 total 471.95 last 15.51) - open/close 9180 (time 1709787332.80 total 482.82 last 16.20) - open/close 9354 (time 1709787343.19 total 493.21 last 16.74) - open/close 9457 (time 1709787353.52 total 503.53 last 9.97) - open/close 9611 (time 1709787364.16 total 514.18 last 14.47) - open/close 9842 (time 1709787374.17 total 524.18 last 23.09) - open/close 9973 (time 1709787384.17 total 534.18 last 13.10) - open/close 10352 (time 1709787394.18 total 544.20 last 37.84) - open/close 10861 (time 1709787404.18 total 554.20 last 50.90) - open/close 11345 (time 1709787414.21 total 564.22 last 48.28) - open/close 11355 (time 1709787425.49 total 575.51 last 0.89) - open/close 11827 (time 1709787435.61 total 585.63 last 46.64) - open/close 12309 (time 1709787445.65 total 595.67 last 48.02) - open/close 12380 (time 1709787458.25 total 608.27 last 5.63) - open/close 12812 (time 1709787468.30 total 618.32 last 42.98) - open/close 13237 (time 1709787478.31 total 628.33 last 42.48) - open/close 13709 (time 1709787488.32 total 638.34 last 47.14) - open/close 14185 (time 1709787498.33 total 648.35 last 47.55) - open/close 14603 (time 1709787508.35 total 658.36 last 41.75) - open/close 14887 (time 1709787526.19 total 676.21 last 15.92) - open/close 15328 (time 1709787536.20 total 686.22 last 44.04) - open/close 15765 (time 1709787546.21 total 696.23 last 43.66) - open/close 16195 (time 1709787556.22 total 706.23 last 42.98) - open/close 16521 (time 1709787566.22 total 716.24 last 32.60) - open/close 16880 (time 1709787576.22 total 726.24 last 35.89) - open/close 17289 (time 1709787586.24 total 736.26 last 40.82) - open/close 17673 (time 1709787596.26 total 746.28 last 38.32) - open/close 18040 (time 1709787606.28 total 756.30 last 36.62) - open/close 18437 (time 1709787616.29 total 766.31 last 39.66) - open/close 18834 (time 1709787626.31 total 776.32 last 39.64) - open/close 19191 (time 1709787636.32 total 786.34 last 35.64) - open/close 19544 (time 1709787646.33 total 796.35 last 35.29) - open/close 19884 (time 1709787656.34 total 806.35 last 33.97) - open/close 20000 (time 1709787659.63 total 809.65 last 35.25) - open/close 20313 (time 1709787669.64 total 819.66 last 31.25) - open/close 20697 (time 1709787679.66 total 829.68 last 38.33) - open/close 21032 (time 1709787689.67 total 839.69 last 33.47) - open/close 21379 (time 1709787699.70 total 849.72 last 34.60) - open/close 21755 (time 1709787709.70 total 859.72 last 37.59) - open/close 22118 (time 1709787719.70 total 869.72 last 36.29) - open/close 22489 (time 1709787729.72 total 879.74 last 37.04) - open/close 22841 (time 1709787739.73 total 889.75 last 35.15) - open/close 23140 (time 1709787749.76 total 899.78 last 29.82) - open/close 23459 (time 1709787759.79 total 909.81 last 31.81) - open/close 23790 (time 1709787769.81 total 919.82 last 33.04) - open/close 24120 (time 1709787779.82 total 929.84 last 32.96) - open/close 24447 (time 1709787789.82 total 939.84 last 32.69) - open/close 24734 (time 1709787799.84 total 949.86 last 28.66) - open/close 25042 (time 1709787809.85 total 959.87 last 30.77) - open/close 25378 (time 1709787819.86 total 969.88 last 33.55) - open/close 25661 (time 1709787829.90 total 979.91 last 28.21) - open/close 25932 (time 1709787839.91 total 989.93 last 27.05) - open/close 26207 (time 1709787849.94 total 999.96 last 27.43) - open/close 26495 (time 1709787859.94 total 1009.96 last 28.80) - open/close 26788 (time 1709787869.97 total 1019.99 last 29.22) - open/close 27062 (time 1709787879.99 total 1030.01 last 27.34) - open/close 27341 (time 1709787890.02 total 1040.04 last 27.81) - open/close 27591 (time 1709787910.73 total 1060.75 last 12.07) - open/close 27861 (time 1709787920.75 total 1070.77 last 26.94) - open/close 28124 (time 1709787930.75 total 1080.77 last 26.30) - open/close 28366 (time 1709787940.79 total 1090.81 last 24.11) - open/close 28651 (time 1709787950.81 total 1100.83 last 28.44) - open/close 28975 (time 1709787960.82 total 1110.84 last 32.38) - open/close 29249 (time 1709787970.85 total 1120.87 last 27.31) - open/close 29572 (time 1709787980.86 total 1130.87 last 32.28) - open/close 29890 (time 1709787990.87 total 1140.89 last 31.76) - open/close 30000 (time 1709787994.74 total 1144.76 last 28.42) - open/close 30280 (time 1709788004.75 total 1154.77 last 27.97) - open/close 30583 (time 1709788014.77 total 1164.79 last 30.22) - open/close 30856 (time 1709788024.79 total 1174.80 last 27.27) - open/close 30983 (time 1709788040.44 total 1190.46 last 8.11) - open/close 31278 (time 1709788050.45 total 1200.47 last 29.46) - open/close 31554 (time 1709788060.45 total 1210.47 last 27.59) - open/close 31834 (time 1709788070.48 total 1220.50 last 27.92) - open/close 32142 (time 1709788080.48 total 1230.50 last 30.80) - open/close 32424 (time 1709788090.52 total 1240.54 last 28.10) - open/close 32701 (time 1709788100.52 total 1250.54 last 27.69) - open/close 32960 (time 1709788110.56 total 1260.58 last 25.80) - open/close 33230 (time 1709788120.57 total 1270.59 last 26.97) - open/close 33469 (time 1709788130.59 total 1280.60 last 23.87) - open/close 33750 (time 1709788140.62 total 1290.64 last 27.99) - open/close 33996 (time 1709788150.65 total 1300.67 last 24.54) - open/close 34249 (time 1709788160.68 total 1310.70 last 25.21) - open/close 34505 (time 1709788170.70 total 1320.72 last 25.56) - open/close 34769 (time 1709788180.70 total 1330.72 last 26.40) - open/close 35017 (time 1709788190.72 total 1340.74 last 24.74) - open/close 35287 (time 1709788200.74 total 1350.76 last 26.96) - open/close 35579 (time 1709788210.77 total 1360.79 last 29.10) - open/close 35853 (time 1709788220.80 total 1370.82 last 27.32) - open/close 36110 (time 1709788230.81 total 1380.83 last 25.67) - open/close 36376 (time 1709788240.82 total 1390.84 last 26.59) - open/close 36619 (time 1709788250.86 total 1400.87 last 24.21) - open/close 36852 (time 1709788260.87 total 1410.89 last 23.27) - open/close 37112 (time 1709788270.88 total 1420.90 last 25.97) - open/close 37363 (time 1709788280.89 total 1430.90 last 25.09) - open/close 37639 (time 1709788290.91 total 1440.93 last 27.53) - open/close 37903 (time 1709788300.93 total 1450.95 last 26.35) - open/close 38164 (time 1709788310.97 total 1460.99 last 26.00) - open/close 38426 (time 1709788320.98 total 1471.00 last 26.17) - open/close 38692 (time 1709788331.03 total 1481.05 last 26.48) - open/close 38946 (time 1709788341.04 total 1491.06 last 25.37) - open/close 39179 (time 1709788351.07 total 1501.09 last 23.23) - open/close 39444 (time 1709788361.10 total 1511.12 last 26.41) - open/close 39689 (time 1709788371.15 total 1521.17 last 24.39) - open/close 39944 (time 1709788381.15 total 1531.17 last 25.50) - open/close 40000 (time 1709788383.36 total 1533.38 last 25.35) - open/close 40253 (time 1709788393.40 total 1543.42 last 25.21) - open/close 40506 (time 1709788403.41 total 1553.43 last 25.27) - open/close 40757 (time 1709788413.43 total 1563.45 last 25.05) - open/close 40994 (time 1709788423.45 total 1573.47 last 23.66) - open/close 41230 (time 1709788433.49 total 1583.50 last 23.51) - open/close 41457 (time 1709788443.52 total 1593.54 last 22.63) - open/close 41676 (time 1709788453.55 total 1603.57 last 21.83) - open/close 41900 (time 1709788463.56 total 1613.58 last 22.37) - open/close 42129 (time 1709788473.58 total 1623.60 last 22.85) - open/close 42353 (time 1709788483.60 total 1633.62 last 22.36) - open/close 42589 (time 1709788493.63 total 1643.64 last 23.55) - open/close 42835 (time 1709788503.65 total 1653.67 last 24.54) - open/close 43073 (time 1709788513.68 total 1663.70 last 23.72) - open/close 43310 (time 1709788523.69 total 1673.71 last 23.67) - open/close 43552 (time 1709788533.73 total 1683.75 last 24.12) - open/close 43801 (time 1709788543.73 total 1693.75 last 24.90) - open/close 44046 (time 1709788553.74 total 1703.76 last 24.47) - open/close 44290 (time 1709788563.74 total 1713.76 last 24.39) - open/close 44533 (time 1709788573.76 total 1723.78 last 24.26) - open/close 44752 (time 1709788583.79 total 1733.81 last 21.83) - open/close 44990 (time 1709788593.83 total 1743.85 last 23.71) - open/close 45223 (time 1709788603.86 total 1753.88 last 23.22) - open/close 45461 (time 1709788613.91 total 1763.92 last 23.70) - open/close 45705 (time 1709788623.91 total 1773.93 last 24.39) - open/close 45945 (time 1709788633.95 total 1783.96 last 23.92) - open/close 46165 (time 1709788643.98 total 1794.00 last 21.92) - open/close 46390 (time 1709788654.00 total 1804.01 last 22.47) - open/close 46615 (time 1709788664.01 total 1814.03 last 22.47) - open/close 46834 (time 1709788674.03 total 1824.05 last 21.85) - open/close 47028 (time 1709788684.07 total 1834.08 last 19.34) - open/close 47233 (time 1709788694.11 total 1844.13 last 20.41) - open/close 47443 (time 1709788704.15 total 1854.17 last 20.91) - open/close 47667 (time 1709788714.18 total 1864.20 last 22.34) - open/close 47883 (time 1709788724.21 total 1874.23 last 21.53) - open/close 48094 (time 1709788734.25 total 1884.27 last 21.01) - open/close 48303 (time 1709788744.30 total 1894.32 last 20.79) - open/close 48509 (time 1709788754.35 total 1904.37 last 20.51) - open/close 48702 (time 1709788764.37 total 1914.39 last 19.26) - open/close 48899 (time 1709788774.40 total 1924.41 last 19.64) - open/close 49099 (time 1709788784.40 total 1934.41 last 20.00) - open/close 49311 (time 1709788794.43 total 1944.45 last 21.12) - open/close 49509 (time 1709788804.45 total 1954.46 last 19.77) - open/close 49719 (time 1709788814.49 total 1964.51 last 20.91) - open/close 49925 (time 1709788824.49 total 1974.51 last 20.59) total: 50000 open/close in 1978.09 seconds: 25.28 ops/second