-----============= acceptance-small: conf-sanity ============----- Wed Mar 6 23:37:01 EST 2024 excepting tests: 32newtarball 110 skipping tests SLOW=no: 45 69 106 111 114 Stopping clients: oleg156-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg156-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg156-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg156-server oleg156-server: oleg156-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing load_modules_local oleg156-server: Loading modules from /home/green/git/lustre-release/lustre oleg156-server: detected 4 online CPUs by sysfs oleg156-server: Force libcfs to create 2 CPU partitions oleg156-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg156-server: oleg156-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg156-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 35b: Continue reconnection retries, if the active server is busy ========================================================== 23:38:02 (1709786282) start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock oleg156-server@tcp:/lustre /mnt/lustre debug=ha conf-sanity.sh test_35b 2024-03-0623h38m15s Set up a fake failnode for the MDS at_max=0 at_max=0 Injecting EBUSY on MDS fail_loc=0x80000136 mdc.lustre-MDT0000-mdc-ffff880137016800.stats=clear Creating a test file and stat it File: '/mnt/lustre/d35b.conf-sanity/f35b.conf-sanity' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934274 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-06 23:38:58.000000000 -0500 Modify: 2024-03-06 23:38:58.000000000 -0500 Change: 2024-03-06 23:38:58.000000000 -0500 Birth: - Stop injecting EBUSY on MDS fail_loc=0 done at_max=600 at_max=600 Debug log: 72 lines, 72 kept, 0 dropped, 0 bad. umount lustre on /mnt/lustre..... Stopping client oleg156-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server unloading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing unload_modules_local modules unloaded. oleg156-server: oleg156-server: tunefs.lustre FATAL: Device lustre-mdt1/mdt1 has not been formatted with mkfs.lustre oleg156-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg156-server: oleg156-server: tunefs.lustre FATAL: Device lustre-ost1/ost1 has not been formatted with mkfs.lustre oleg156-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg156-server: oleg156-server: tunefs.lustre FATAL: Device lustre-ost2/ost2 has not been formatted with mkfs.lustre oleg156-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 19 checking for existing Lustre data: not found tunefs failed, reformatting instead Stopping clients: oleg156-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg156-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 oleg156-server: oleg156-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing load_modules_local oleg156-server: Loading modules from /home/green/git/lustre-release/lustre oleg156-server: detected 4 online CPUs by sysfs oleg156-server: Force libcfs to create 2 CPU partitions oleg156-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg156-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg156-server: oleg156-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg156-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server PASS 35b (116s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 23:39:59 (1709786399) start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock oleg156-server@tcp:/lustre /mnt/lustre copying 10 files to /mnt/lustre/d38.conf-sanity tar: Removing leading `/' from member names umount lustre on /mnt/lustre..... Stopping client oleg156-client.virtnet /mnt/lustre (opts:) osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=44 stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server delete lov_objid file on MDS 000000 43 000008 start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock oleg156-server@tcp:/lustre /mnt/lustre osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=2 umount lustre on /mnt/lustre..... Stopping client oleg156-client.virtnet /mnt/lustre (opts:) stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server 000000 2 000008 oleg156-server: 8+0 records in oleg156-server: 8+0 records out oleg156-server: 4096 bytes (4.1 kB) copied, 0.000653825 s, 6.3 MB/s start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock oleg156-server@tcp:/lustre /mnt/lustre osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=35 umount lustre on /mnt/lustre..... Stopping client oleg156-client.virtnet /mnt/lustre (opts:) stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server 000000 34 000008 files compared the same umount lustre on /mnt/lustre..... stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server unloading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg156-client: oleg156-client: ssh exited with exit code 2 pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 pdsh@oleg156-client: oleg156-client: ssh exited with exit code 2 pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 PASS 38 (58s) error: set_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: setting 'debug_raw_pointers'='0': No such file or directory oleg156-server: error: set_param: param_path 'debug_raw_pointers': No such file or directory oleg156-server: error: set_param: setting 'debug_raw_pointers'='0': No such file or directory error: get_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: setting 'debug_raw_pointers'='Y': No such file or directory oleg156-server: error: get_param: param_path 'debug_raw_pointers': No such file or directory pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 oleg156-server: error: set_param: param_path 'debug_raw_pointers': No such file or directory oleg156-server: error: set_param: setting 'debug_raw_pointers'='Y': No such file or directory == conf-sanity test 51: Verify that mdt_reint handles RMF_MDT_MD correctly when an OST is added ========================================================== 23:40:59 (1709786459) Stopping clients: oleg156-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg156-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 oleg156-server: oleg156-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing load_modules_local oleg156-server: Loading modules from /home/green/git/lustre-release/lustre oleg156-server: detected 4 online CPUs by sysfs oleg156-server: Force libcfs to create 2 CPU partitions oleg156-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg156-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg156-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock oleg156-server@tcp:/lustre /mnt/lustre setup single mount lustre success fail_loc=0x142 start ost2 service on oleg156-server Starting ost2: -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-ost2/ost2 Started lustre-OST0001 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec stop ost2 service on oleg156-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg156-server umount lustre on /mnt/lustre..... Stopping client oleg156-client.virtnet /mnt/lustre (opts:-f) stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server unloading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing unload_modules_local modules unloaded. oleg156-server: oleg156-server: tunefs.lustre FATAL: Device lustre-mdt1/mdt1 has not been formatted with mkfs.lustre oleg156-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg156-server: oleg156-server: tunefs.lustre FATAL: Device lustre-ost1/ost1 has not been formatted with mkfs.lustre oleg156-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg156-server: oleg156-server: tunefs.lustre FATAL: Device lustre-ost2/ost2 has not been formatted with mkfs.lustre oleg156-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 19 checking for existing Lustre data: not found tunefs failed, reformatting instead Stopping clients: oleg156-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg156-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg156-client: oleg156-server: ssh exited with exit code 2 oleg156-server: oleg156-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg156-server' oleg156-server: oleg156-server.virtnet: executing load_modules_local oleg156-server: Loading modules from /home/green/git/lustre-release/lustre oleg156-server: detected 4 online CPUs by sysfs oleg156-server: Force libcfs to create 2 CPU partitions oleg156-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg156-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg156-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg156-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg156-server: oleg156-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg156-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg156-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg156-server stop mds service on oleg156-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg156-server PASS 51 (138s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 23:43:19 (1709786599) start mds service on oleg156-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg156-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg156-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg156-server: oleg156-server.virtnet: executing set_default_debug -1 all pdsh@oleg156-client: oleg156-server: ssh exited with exit code 1 Started lustre-OST0000 oleg156-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg156-client.virtnet: -o user_xattr,flock oleg156-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 193 (time 1709786629.48 total 10.63 last 18.16) - open/close 448 (time 1709786639.56 total 20.70 last 25.31) - open/close 818 (time 1709786649.56 total 30.70 last 37.00) - open/close 1135 (time 1709786659.63 total 40.77 last 31.49) - open/close 1382 (time 1709786670.00 total 51.14 last 23.81) - open/close 1585 (time 1709786680.01 total 61.15 last 20.28) - open/close 1855 (time 1709786690.03 total 71.17 last 26.95) - open/close 2054 (time 1709786700.93 total 82.07 last 18.25) - open/close 2255 (time 1709786711.90 total 93.04 last 18.32) - open/close 2525 (time 1709786722.59 total 103.73 last 25.27) - open/close 2696 (time 1709786733.14 total 114.29 last 16.20) - open/close 2978 (time 1709786743.15 total 124.29 last 28.18) - open/close 3114 (time 1709786753.26 total 134.41 last 13.45) - open/close 3374 (time 1709786763.27 total 144.41 last 25.98) - open/close 3499 (time 1709786774.15 total 155.29 last 11.49) - open/close 3637 (time 1709786784.16 total 165.30 last 13.79) - open/close 3926 (time 1709786794.55 total 175.69 last 27.82) - open/close 4101 (time 1709786805.43 total 186.57 last 16.08) - open/close 4341 (time 1709786815.93 total 197.07 last 22.85) - open/close 4662 (time 1709786826.09 total 207.24 last 31.58) - open/close 4760 (time 1709786837.74 total 218.88 last 8.42) - open/close 4911 (time 1709786848.63 total 229.78 last 13.86) - open/close 5075 (time 1709786858.70 total 239.84 last 16.29) - open/close 5306 (time 1709786868.71 total 249.85 last 23.07) - open/close 5492 (time 1709786878.74 total 259.88 last 18.55) - open/close 5648 (time 1709786888.74 total 269.89 last 15.59) - open/close 5819 (time 1709786898.75 total 279.89 last 17.10) - open/close 6070 (time 1709786909.26 total 290.40 last 23.88) - open/close 6225 (time 1709786920.23 total 301.37 last 14.13) - open/close 6436 (time 1709786931.15 total 312.29 last 19.31) - open/close 6608 (time 1709786942.04 total 323.18 last 15.79) - open/close 6727 (time 1709786952.44 total 333.58 last 11.45) - open/close 6940 (time 1709786962.45 total 343.59 last 21.27) - open/close 7141 (time 1709786972.46 total 353.60 last 20.08) - open/close 7414 (time 1709786982.47 total 363.61 last 27.27) - open/close 7633 (time 1709786992.48 total 373.62 last 21.88) - open/close 7923 (time 1709787004.78 total 385.92 last 23.58) - open/close 8136 (time 1709787015.68 total 396.82 last 19.54) - open/close 8301 (time 1709787026.61 total 407.75 last 15.10) - open/close 8530 (time 1709787036.62 total 417.76 last 22.89) - open/close 8761 (time 1709787046.90 total 428.04 last 22.47) - open/close 8905 (time 1709787056.90 total 438.04 last 14.40) - open/close 9102 (time 1709787066.96 total 448.10 last 19.59) - open/close 9277 (time 1709787076.97 total 458.11 last 17.48) - open/close 9456 (time 1709787086.98 total 468.12 last 17.88) - open/close 9981 (time 1709787096.98 total 478.12 last 52.50) - open/close 10379 (time 1709787115.84 total 496.98 last 21.10) - open/close 10918 (time 1709787125.86 total 507.00 last 53.81) - open/close 11404 (time 1709787135.87 total 517.01 last 48.54) - open/close 11838 (time 1709787145.87 total 527.01 last 43.38) - open/close 12293 (time 1709787155.88 total 537.02 last 45.47) - open/close 12741 (time 1709787165.89 total 547.03 last 44.75) - open/close 13171 (time 1709787175.89 total 557.03 last 43.00) - open/close 13621 (time 1709787185.90 total 567.05 last 44.93) - open/close 14007 (time 1709787195.91 total 577.05 last 38.59) - open/close 14393 (time 1709787205.91 total 587.05 last 38.60) - open/close 14789 (time 1709787215.93 total 597.07 last 39.52) - open/close 15125 (time 1709787225.94 total 607.08 last 33.56) - open/close 15495 (time 1709787235.96 total 617.10 last 36.95) - open/close 15848 (time 1709787245.98 total 627.12 last 35.23) - open/close 15931 (time 1709787259.10 total 640.24 last 6.33) - open/close 16355 (time 1709787269.13 total 650.27 last 42.25) - open/close 16777 (time 1709787279.13 total 660.28 last 42.20) - open/close 17157 (time 1709787289.15 total 670.30 last 37.92) - open/close 17540 (time 1709787299.16 total 680.30 last 38.27) - open/close 17914 (time 1709787309.20 total 690.34 last 37.25) - open/close 18240 (time 1709787319.21 total 700.35 last 32.59) - open/close 18484 (time 1709787338.02 total 719.17 last 12.97) - open/close 18806 (time 1709787348.04 total 729.18 last 32.14) - open/close 19122 (time 1709787358.07 total 739.21 last 31.52) - open/close 19424 (time 1709787368.08 total 749.22 last 30.17) - open/close 19727 (time 1709787378.10 total 759.24 last 30.22) - open/close 20000 (time 1709787387.24 total 768.38 last 29.87) - open/close 20286 (time 1709787397.28 total 778.42 last 28.50) - open/close 20557 (time 1709787407.32 total 788.46 last 26.98) - open/close 20848 (time 1709787417.33 total 798.47 last 29.07) - open/close 21170 (time 1709787427.36 total 808.50 last 32.12) - open/close 21460 (time 1709787437.38 total 818.52 last 28.94) - open/close 21754 (time 1709787447.39 total 828.54 last 29.35) - open/close 22053 (time 1709787457.41 total 838.55 last 29.86) - open/close 22310 (time 1709787467.46 total 848.60 last 25.58) - open/close 22563 (time 1709787477.50 total 858.64 last 25.20) - open/close 22841 (time 1709787487.53 total 868.67 last 27.72) - open/close 23109 (time 1709787497.54 total 878.68 last 26.76) - open/close 23333 (time 1709787507.56 total 888.71 last 22.35) - open/close 23588 (time 1709787517.60 total 898.74 last 25.41) - open/close 23864 (time 1709787527.62 total 908.76 last 27.55) - open/close 24129 (time 1709787537.65 total 918.79 last 26.42) - open/close 24356 (time 1709787547.66 total 928.80 last 22.67) - open/close 24597 (time 1709787557.67 total 938.81 last 24.07) - open/close 24785 (time 1709787567.72 total 948.86 last 18.72) - open/close 25031 (time 1709787577.73 total 958.87 last 24.57) - open/close 25284 (time 1709787587.78 total 968.92 last 25.18) - open/close 25539 (time 1709787597.80 total 978.94 last 25.44) - open/close 25627 (time 1709787612.38 total 993.52 last 6.04) - open/close 25875 (time 1709787622.38 total 1003.53 last 24.78) - open/close 26106 (time 1709787632.39 total 1013.53 last 23.10) - open/close 26387 (time 1709787642.39 total 1023.53 last 28.09) - open/close 26550 (time 1709787659.68 total 1040.82 last 9.43) - open/close 26605 (time 1709787672.71 total 1053.85 last 4.22) - open/close 26851 (time 1709787682.71 total 1063.85 last 24.59) - open/close 27099 (time 1709787692.75 total 1073.90 last 24.69) - open/close 27361 (time 1709787702.80 total 1083.95 last 26.07) - open/close 27615 (time 1709787712.82 total 1093.96 last 25.35) - open/close 27875 (time 1709787722.85 total 1104.00 last 25.92) - open/close 28142 (time 1709787732.91 total 1114.05 last 26.56) - open/close 28394 (time 1709787742.93 total 1124.07 last 25.14) - open/close 28639 (time 1709787752.94 total 1134.08 last 24.48) - open/close 28869 (time 1709787762.95 total 1144.09 last 22.98) - open/close 29112 (time 1709787772.98 total 1154.12 last 24.22) - open/close 29347 (time 1709787783.00 total 1164.14 last 23.46) - open/close 29581 (time 1709787793.04 total 1174.18 last 23.31) - open/close 29796 (time 1709787803.04 total 1184.18 last 21.49) - open/close 30000 (time 1709787811.15 total 1192.29 last 25.17) - open/close 30260 (time 1709787821.15 total 1202.30 last 25.98) - open/close 30504 (time 1709787831.19 total 1212.33 last 24.31) - open/close 30720 (time 1709787841.23 total 1222.37 last 21.52) - open/close 30936 (time 1709787851.27 total 1232.41 last 21.52) - open/close 31163 (time 1709787861.33 total 1242.47 last 22.56) - open/close 31375 (time 1709787871.36 total 1252.50 last 21.14) - open/close 31591 (time 1709787881.39 total 1262.53 last 21.52) - open/close 31796 (time 1709787891.43 total 1272.57 last 20.42) - open/close 32018 (time 1709787901.46 total 1282.60 last 22.14) - open/close 32232 (time 1709787911.49 total 1292.63 last 21.32) - open/close 32436 (time 1709787921.53 total 1302.67 last 20.33) - open/close 32633 (time 1709787931.53 total 1312.67 last 19.69) - open/close 32853 (time 1709787941.58 total 1322.72 last 21.88) - open/close 33078 (time 1709787951.62 total 1332.76 last 22.43) - open/close 33321 (time 1709787961.62 total 1342.76 last 24.29) - open/close 33545 (time 1709787971.63 total 1352.77 last 22.39) - open/close 33772 (time 1709787981.66 total 1362.80 last 22.63) - open/close 34013 (time 1709787991.68 total 1372.82 last 24.05) - open/close 34240 (time 1709788001.69 total 1382.84 last 22.66) - open/close 34462 (time 1709788011.75 total 1392.90 last 22.07) - open/close 34679 (time 1709788021.78 total 1402.92 last 21.65) - open/close 34911 (time 1709788031.82 total 1412.96 last 23.11) - open/close 35138 (time 1709788041.83 total 1422.97 last 22.66) - open/close 35367 (time 1709788051.88 total 1433.02 last 22.79) - open/close 35597 (time 1709788061.91 total 1443.06 last 22.92) - open/close 35826 (time 1709788071.92 total 1453.06 last 22.88) - open/close 36069 (time 1709788081.98 total 1463.12 last 24.16) - open/close 36298 (time 1709788092.02 total 1473.16 last 22.81) - open/close 36533 (time 1709788102.05 total 1483.19 last 23.43) - open/close 36748 (time 1709788112.07 total 1493.21 last 21.47) - open/close 36979 (time 1709788122.08 total 1503.22 last 23.08) - open/close 37190 (time 1709788132.12 total 1513.26 last 21.01) - open/close 37419 (time 1709788142.12 total 1523.27 last 22.89) - open/close 37627 (time 1709788152.19 total 1533.33 last 20.66) - open/close 37834 (time 1709788162.19 total 1543.34 last 20.69) - open/close 38044 (time 1709788172.21 total 1553.35 last 20.97) - open/close 38257 (time 1709788182.22 total 1563.36 last 21.27) - open/close 38467 (time 1709788192.24 total 1573.38 last 20.96) - open/close 38699 (time 1709788202.27 total 1583.41 last 23.14) - open/close 38947 (time 1709788212.29 total 1593.43 last 24.76) - open/close 39184 (time 1709788222.30 total 1603.44 last 23.67) - open/close 39301 (time 1709788238.53 total 1619.67 last 7.21) - open/close 39529 (time 1709788248.56 total 1629.70 last 22.75) - open/close 39732 (time 1709788258.58 total 1639.72 last 20.25) - open/close 39945 (time 1709788268.61 total 1649.75 last 21.25) - open/close 40000 (time 1709788271.38 total 1652.52 last 19.84) - open/close 40225 (time 1709788281.41 total 1662.55 last 22.43) - open/close 40458 (time 1709788291.42 total 1672.56 last 23.28) - open/close 40690 (time 1709788301.45 total 1682.59 last 23.13) - open/close 40920 (time 1709788311.46 total 1692.60 last 22.99) - open/close 41142 (time 1709788321.49 total 1702.63 last 22.12) - open/close 41372 (time 1709788331.52 total 1712.66 last 22.94) - open/close 41591 (time 1709788341.52 total 1722.66 last 21.88) - open/close 41808 (time 1709788351.53 total 1732.67 last 21.70) - open/close 42038 (time 1709788361.56 total 1742.70 last 22.91) - open/close 42254 (time 1709788371.59 total 1752.73 last 21.55) - open/close 42475 (time 1709788381.61 total 1762.76 last 22.04) - open/close 42688 (time 1709788391.64 total 1772.78 last 21.24) - open/close 42906 (time 1709788401.64 total 1782.79 last 21.79) - open/close 43133 (time 1709788411.67 total 1792.81 last 22.64) - open/close 43333 (time 1709788421.72 total 1802.86 last 19.89) - open/close 43527 (time 1709788431.75 total 1812.89 last 19.35) - open/close 43731 (time 1709788441.78 total 1822.92 last 20.35) - open/close 43927 (time 1709788451.81 total 1832.96 last 19.53) - open/close 44127 (time 1709788461.83 total 1842.98 last 19.96) - open/close 44315 (time 1709788471.88 total 1853.02 last 18.72) - open/close 44519 (time 1709788481.92 total 1863.06 last 20.31) - open/close 44723 (time 1709788491.95 total 1873.10 last 20.34) - open/close 44936 (time 1709788501.96 total 1883.10 last 21.28) - open/close 45149 (time 1709788511.99 total 1893.13 last 21.25) - open/close 45364 (time 1709788522.04 total 1903.18 last 21.39) - open/close 45577 (time 1709788532.07 total 1913.21 last 21.23) - open/close 45797 (time 1709788542.09 total 1923.23 last 21.95) - open/close 46012 (time 1709788552.11 total 1933.25 last 21.47) - open/close 46226 (time 1709788562.12 total 1943.27 last 21.36) - open/close 46443 (time 1709788572.13 total 1953.27 last 21.69) - open/close 46626 (time 1709788582.13 total 1963.27 last 18.29) - open/close 46833 (time 1709788592.15 total 1973.29 last 20.66) - open/close 47037 (time 1709788602.16 total 1983.30 last 20.38) - open/close 47233 (time 1709788612.16 total 1993.31 last 19.59) - open/close 47444 (time 1709788622.17 total 2003.31 last 21.09) - open/close 47676 (time 1709788632.18 total 2013.32 last 23.18) - open/close 47884 (time 1709788642.21 total 2023.35 last 20.74) - open/close 48087 (time 1709788652.22 total 2033.36 last 20.27) - open/close 48289 (time 1709788662.23 total 2043.37 last 20.19) - open/close 48491 (time 1709788672.27 total 2053.42 last 20.11) - open/close 48667 (time 1709788682.31 total 2063.45 last 17.54) - open/close 48845 (time 1709788692.33 total 2073.48 last 17.75) - open/close 49028 (time 1709788702.36 total 2083.51 last 18.24) - open/close 49230 (time 1709788712.41 total 2093.55 last 20.11) - open/close 49431 (time 1709788722.42 total 2103.56 last 20.08) - open/close 49631 (time 1709788732.42 total 2113.56 last 20.00) - open/close 49825 (time 1709788742.42 total 2123.56 last 19.39) total: 50000 open/close in 2132.99 seconds: 23.44 ops/second