-----============= acceptance-small: conf-sanity ============----- Wed Mar 6 23:39:50 EST 2024 excepting tests: 32newtarball 110 skipping tests SLOW=no: 45 69 106 111 114 Stopping clients: oleg132-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg132-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg132-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg132-server oleg132-server: oleg132-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing load_modules_local oleg132-server: Loading modules from /home/green/git/lustre-release/lustre oleg132-server: detected 4 online CPUs by sysfs oleg132-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg132-server: oleg132-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg132-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 35b: Continue reconnection retries, if the active server is busy ========================================================== 23:40:49 (1709786449) start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock oleg132-server@tcp:/lustre /mnt/lustre debug=ha conf-sanity.sh test_35b 2024-03-0623h41m03s Set up a fake failnode for the MDS at_max=0 at_max=0 Injecting EBUSY on MDS fail_loc=0x80000136 mdc.lustre-MDT0000-mdc-ffff8800ab2f4000.stats=clear Creating a test file and stat it File: '/mnt/lustre/d35b.conf-sanity/f35b.conf-sanity' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934274 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-06 23:41:46.000000000 -0500 Modify: 2024-03-06 23:41:46.000000000 -0500 Change: 2024-03-06 23:41:46.000000000 -0500 Birth: - Stop injecting EBUSY on MDS fail_loc=0 done at_max=600 at_max=600 Debug log: 54 lines, 54 kept, 0 dropped, 0 bad. umount lustre on /mnt/lustre..... Stopping client oleg132-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server unloading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing unload_modules_local modules unloaded. oleg132-server: oleg132-server: tunefs.lustre FATAL: Device lustre-mdt1/mdt1 has not been formatted with mkfs.lustre oleg132-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg132-server: oleg132-server: tunefs.lustre FATAL: Device lustre-ost1/ost1 has not been formatted with mkfs.lustre oleg132-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg132-server: oleg132-server: tunefs.lustre FATAL: Device lustre-ost2/ost2 has not been formatted with mkfs.lustre oleg132-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 19 checking for existing Lustre data: not found tunefs failed, reformatting instead Stopping clients: oleg132-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg132-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 2 oleg132-server: oleg132-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing load_modules_local oleg132-server: Loading modules from /home/green/git/lustre-release/lustre oleg132-server: detected 4 online CPUs by sysfs oleg132-server: Force libcfs to create 2 CPU partitions oleg132-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg132-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg132-server: oleg132-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg132-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server PASS 35b (119s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 23:42:50 (1709786570) start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock oleg132-server@tcp:/lustre /mnt/lustre copying 10 files to /mnt/lustre/d38.conf-sanity tar: Removing leading `/' from member names umount lustre on /mnt/lustre..... Stopping client oleg132-client.virtnet /mnt/lustre (opts:) osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=44 stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server delete lov_objid file on MDS 000000 43 000008 start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock oleg132-server@tcp:/lustre /mnt/lustre osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=2 umount lustre on /mnt/lustre..... Stopping client oleg132-client.virtnet /mnt/lustre (opts:) stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server 000000 2 000008 oleg132-server: 8+0 records in oleg132-server: 8+0 records out oleg132-server: 4096 bytes (4.1 kB) copied, 0.000671855 s, 6.1 MB/s start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock oleg132-server@tcp:/lustre /mnt/lustre osp.lustre-OST0000-osc-MDT0000.prealloc_next_id=35 umount lustre on /mnt/lustre..... Stopping client oleg132-client.virtnet /mnt/lustre (opts:) stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server 000000 34 000008 files compared the same umount lustre on /mnt/lustre..... stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server unloading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg132-client: oleg132-client: ssh exited with exit code 2 pdsh@oleg132-client: oleg132-server: ssh exited with exit code 2 pdsh@oleg132-client: oleg132-client: ssh exited with exit code 2 pdsh@oleg132-client: oleg132-server: ssh exited with exit code 2 PASS 38 (71s) error: set_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: setting 'debug_raw_pointers'='0': No such file or directory oleg132-server: error: set_param: param_path 'debug_raw_pointers': No such file or directory oleg132-server: error: set_param: setting 'debug_raw_pointers'='0': No such file or directory error: get_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: param_path 'debug_raw_pointers': No such file or directory error: set_param: setting 'debug_raw_pointers'='Y': No such file or directory oleg132-server: error: get_param: param_path 'debug_raw_pointers': No such file or directory pdsh@oleg132-client: oleg132-server: ssh exited with exit code 2 oleg132-server: error: set_param: param_path 'debug_raw_pointers': No such file or directory oleg132-server: error: set_param: setting 'debug_raw_pointers'='Y': No such file or directory == conf-sanity test 51: Verify that mdt_reint handles RMF_MDT_MD correctly when an OST is added ========================================================== 23:44:03 (1709786643) Stopping clients: oleg132-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg132-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 2 oleg132-server: oleg132-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing load_modules_local oleg132-server: Loading modules from /home/green/git/lustre-release/lustre oleg132-server: detected 4 online CPUs by sysfs oleg132-server: Force libcfs to create 2 CPU partitions oleg132-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg132-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock oleg132-server@tcp:/lustre /mnt/lustre setup single mount lustre success fail_loc=0x142 start ost2 service on oleg132-server Starting ost2: -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-ost2/ost2 Started lustre-OST0001 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec stop ost2 service on oleg132-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg132-server umount lustre on /mnt/lustre..... Stopping client oleg132-client.virtnet /mnt/lustre (opts:-f) stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server unloading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing unload_modules_local modules unloaded. oleg132-server: oleg132-server: tunefs.lustre FATAL: Device lustre-mdt1/mdt1 has not been formatted with mkfs.lustre oleg132-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg132-server: oleg132-server: tunefs.lustre FATAL: Device lustre-ost1/ost1 has not been formatted with mkfs.lustre oleg132-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 19 checking for existing Lustre data: not found oleg132-server: oleg132-server: tunefs.lustre FATAL: Device lustre-ost2/ost2 has not been formatted with mkfs.lustre oleg132-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 19 checking for existing Lustre data: not found tunefs failed, reformatting instead Stopping clients: oleg132-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg132-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg132-client: oleg132-server: ssh exited with exit code 2 oleg132-server: oleg132-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing load_modules_local oleg132-server: Loading modules from /home/green/git/lustre-release/lustre oleg132-server: detected 4 online CPUs by sysfs oleg132-server: Force libcfs to create 2 CPU partitions oleg132-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg132-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg132-server: oleg132-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg132-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server PASS 51 (171s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 23:46:58 (1709786818) start mds service on oleg132-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg132-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock oleg132-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 265 (time 1709786847.27 total 10.10 last 26.23) - open/close 377 (time 1709786857.80 total 20.64 last 10.63) - open/close 698 (time 1709786867.81 total 30.65 last 32.08) - open/close 946 (time 1709786878.71 total 41.54 last 22.76) - open/close 1094 (time 1709786889.02 total 51.85 last 14.35) - open/close 1266 (time 1709786899.77 total 62.61 last 15.99) - open/close 1572 (time 1709786909.94 total 72.77 last 30.11) - open/close 1754 (time 1709786920.10 total 82.93 last 17.91) - open/close 1898 (time 1709786930.11 total 92.94 last 14.39) - open/close 2057 (time 1709786941.56 total 104.40 last 13.88) - open/close 2272 (time 1709786951.56 total 114.40 last 21.50) - open/close 2475 (time 1709786962.33 total 125.17 last 18.86) - open/close 2651 (time 1709786972.34 total 135.17 last 17.59) - open/close 2847 (time 1709786982.83 total 145.67 last 18.68) - open/close 3074 (time 1709786992.86 total 155.69 last 22.64) - open/close 3337 (time 1709787002.86 total 165.69 last 26.29) - open/close 3628 (time 1709787013.13 total 175.97 last 28.33) - open/close 3880 (time 1709787023.71 total 186.54 last 23.83) - open/close 4086 (time 1709787034.21 total 197.04 last 19.62) - open/close 4367 (time 1709787044.92 total 207.75 last 26.23) - open/close 4492 (time 1709787056.89 total 219.72 last 10.44) - open/close 4780 (time 1709787067.61 total 230.45 last 26.85) - open/close 4973 (time 1709787078.40 total 241.23 last 17.90) - open/close 5224 (time 1709787088.41 total 251.25 last 25.06) - open/close 5409 (time 1709787098.86 total 261.70 last 17.70) - open/close 5627 (time 1709787109.49 total 272.33 last 20.51) - open/close 5786 (time 1709787120.19 total 283.02 last 14.87) - open/close 5961 (time 1709787130.20 total 293.03 last 17.49) - open/close 6118 (time 1709787140.86 total 303.69 last 14.73) - open/close 6279 (time 1709787150.88 total 313.72 last 16.06) - open/close 6523 (time 1709787161.25 total 324.09 last 23.54) - open/close 6667 (time 1709787171.26 total 334.09 last 14.39) - open/close 6880 (time 1709787181.39 total 344.22 last 21.02) - open/close 7000 (time 1709787193.16 total 356.00 last 10.19) - open/close 7128 (time 1709787203.31 total 366.14 last 12.62) - open/close 7269 (time 1709787213.68 total 376.51 last 13.60) - open/close 7392 (time 1709787224.36 total 387.19 last 11.52) - open/close 7755 (time 1709787234.37 total 397.20 last 36.25) - open/close 7930 (time 1709787244.37 total 407.21 last 17.49) - open/close 8131 (time 1709787254.38 total 417.22 last 20.08) - open/close 8292 (time 1709787264.40 total 427.23 last 16.08) - open/close 8430 (time 1709787274.89 total 437.72 last 13.15) - open/close 8605 (time 1709787284.99 total 447.82 last 17.33) - open/close 8805 (time 1709787295.00 total 457.83 last 19.98) - open/close 8911 (time 1709787305.69 total 468.53 last 9.91) - open/close 9120 (time 1709787316.66 total 479.50 last 19.05) - open/close 9201 (time 1709787326.70 total 489.54 last 8.07) - open/close 9361 (time 1709787337.11 total 499.94 last 15.38) - open/close 9507 (time 1709787347.62 total 510.46 last 13.89) - open/close 9727 (time 1709787359.83 total 522.67 last 18.02) - open/close 9942 (time 1709787369.85 total 532.68 last 21.47) - open/close 10015 (time 1709787382.47 total 545.30 last 5.78) - open/close 10517 (time 1709787392.47 total 555.31 last 50.16) - open/close 11031 (time 1709787402.51 total 565.35 last 51.21) - open/close 11550 (time 1709787412.52 total 575.35 last 51.86) - open/close 12064 (time 1709787422.54 total 585.37 last 51.31) - open/close 12572 (time 1709787432.55 total 595.39 last 50.72) - open/close 13056 (time 1709787442.58 total 605.41 last 48.27) - open/close 13486 (time 1709787452.60 total 615.44 last 42.91) - open/close 13815 (time 1709787471.40 total 634.23 last 17.50) - open/close 14206 (time 1709787481.40 total 644.24 last 39.07) - open/close 14680 (time 1709787491.40 total 654.24 last 47.39) - open/close 15101 (time 1709787501.44 total 664.27 last 41.96) - open/close 15536 (time 1709787511.45 total 674.29 last 43.44) - open/close 15565 (time 1709787523.30 total 686.13 last 2.45) - open/close 15983 (time 1709787533.36 total 696.20 last 41.53) - open/close 16406 (time 1709787543.39 total 706.22 last 42.18) - open/close 16809 (time 1709787553.39 total 716.22 last 40.30) - open/close 17180 (time 1709787563.39 total 726.23 last 37.09) - open/close 17554 (time 1709787573.41 total 736.25 last 37.32) - open/close 17945 (time 1709787583.42 total 746.25 last 39.09) - open/close 18330 (time 1709787593.43 total 756.26 last 38.45) - open/close 18675 (time 1709787603.46 total 766.30 last 34.39) - open/close 19099 (time 1709787613.47 total 776.31 last 42.35) - open/close 19459 (time 1709787623.48 total 786.32 last 35.96) - open/close 19779 (time 1709787633.51 total 796.34 last 31.93) - open/close 20000 (time 1709787639.07 total 801.90 last 39.75) - open/close 20329 (time 1709787649.08 total 811.92 last 32.85) - open/close 20668 (time 1709787659.09 total 821.93 last 33.87) - open/close 20983 (time 1709787669.10 total 831.93 last 31.48) - open/close 21317 (time 1709787679.15 total 841.98 last 33.24) - open/close 21635 (time 1709787689.17 total 852.01 last 31.72) - open/close 21984 (time 1709787699.20 total 862.03 last 34.81) - open/close 22359 (time 1709787709.28 total 872.12 last 37.18) - open/close 22737 (time 1709787719.30 total 882.14 last 37.72) - open/close 23109 (time 1709787729.33 total 892.16 last 37.11) - open/close 23440 (time 1709787739.33 total 902.17 last 33.09) - open/close 23770 (time 1709787749.36 total 912.19 last 32.93) - open/close 24083 (time 1709787759.36 total 922.20 last 31.27) - open/close 24396 (time 1709787769.38 total 932.21 last 31.25) - open/close 24712 (time 1709787779.39 total 942.23 last 31.55) - open/close 25008 (time 1709787789.41 total 952.25 last 29.55) - open/close 25289 (time 1709787799.43 total 962.27 last 28.04) - open/close 25610 (time 1709787809.46 total 972.29 last 32.03) - open/close 25932 (time 1709787819.47 total 982.31 last 32.15) - open/close 26231 (time 1709787829.48 total 992.32 last 29.86) - open/close 26505 (time 1709787839.50 total 1002.34 last 27.35) - open/close 26793 (time 1709787849.55 total 1012.39 last 28.66) - open/close 27058 (time 1709787859.69 total 1022.53 last 26.13) - open/close 27349 (time 1709787869.72 total 1032.55 last 29.03) - open/close 27637 (time 1709787879.72 total 1042.55 last 28.79) - open/close 27926 (time 1709787889.74 total 1052.57 last 28.84) - open/close 28198 (time 1709787899.76 total 1062.59 last 27.15) - open/close 28470 (time 1709787909.77 total 1072.60 last 27.18) - open/close 28740 (time 1709787919.77 total 1082.60 last 26.99) - open/close 29003 (time 1709787929.79 total 1092.63 last 26.23) - open/close 29258 (time 1709787939.83 total 1102.67 last 25.40) - open/close 29417 (time 1709787957.72 total 1120.55 last 8.89) - open/close 29711 (time 1709787967.73 total 1130.56 last 29.36) - open/close 30000 (time 1709787977.30 total 1140.14 last 30.19) - open/close 30304 (time 1709787987.32 total 1150.16 last 30.33) - open/close 30607 (time 1709787997.34 total 1160.17 last 30.27) - open/close 30889 (time 1709788007.36 total 1170.19 last 28.14) - open/close 31183 (time 1709788017.38 total 1180.22 last 29.32) - open/close 31456 (time 1709788027.40 total 1190.24 last 27.25) - open/close 31744 (time 1709788037.41 total 1200.24 last 28.79) - open/close 32018 (time 1709788047.43 total 1210.27 last 27.32) - open/close 32301 (time 1709788057.46 total 1220.30 last 28.23) - open/close 32593 (time 1709788067.48 total 1230.31 last 29.16) - open/close 32875 (time 1709788077.49 total 1240.32 last 28.17) - open/close 33164 (time 1709788087.52 total 1250.35 last 28.81) - open/close 33452 (time 1709788097.53 total 1260.37 last 28.75) - open/close 33717 (time 1709788107.54 total 1270.37 last 26.49) - open/close 33999 (time 1709788117.57 total 1280.40 last 28.12) - open/close 34260 (time 1709788127.57 total 1290.41 last 26.08) - open/close 34545 (time 1709788137.57 total 1300.41 last 28.50) - open/close 34810 (time 1709788147.62 total 1310.45 last 26.39) - open/close 35061 (time 1709788157.64 total 1320.47 last 25.04) - open/close 35308 (time 1709788167.68 total 1330.51 last 24.60) - open/close 35568 (time 1709788177.72 total 1340.56 last 25.89) - open/close 35822 (time 1709788187.74 total 1350.58 last 25.34) - open/close 36072 (time 1709788197.78 total 1360.62 last 24.91) - open/close 36369 (time 1709788207.81 total 1370.65 last 29.61) - open/close 36652 (time 1709788217.81 total 1380.65 last 28.30) - open/close 36909 (time 1709788227.84 total 1390.68 last 25.63) - open/close 37190 (time 1709788237.86 total 1400.70 last 28.04) - open/close 37377 (time 1709788256.03 total 1418.87 last 10.29) - open/close 37615 (time 1709788266.04 total 1428.88 last 23.78) - open/close 37863 (time 1709788276.07 total 1438.91 last 24.73) - open/close 38114 (time 1709788286.10 total 1448.94 last 25.02) - open/close 38380 (time 1709788296.12 total 1458.96 last 26.54) - open/close 38644 (time 1709788306.16 total 1469.00 last 26.30) - open/close 38881 (time 1709788316.18 total 1479.01 last 23.67) - open/close 39139 (time 1709788326.18 total 1489.01 last 25.80) - open/close 39394 (time 1709788336.18 total 1499.02 last 25.49) - open/close 39633 (time 1709788346.20 total 1509.03 last 23.86) - open/close 39886 (time 1709788356.22 total 1519.06 last 25.24) - open/close 40000 (time 1709788360.48 total 1523.31 last 26.81) - open/close 40248 (time 1709788370.49 total 1533.33 last 24.77) - open/close 40488 (time 1709788380.49 total 1543.33 last 23.99) - open/close 40735 (time 1709788390.53 total 1553.37 last 24.60) - open/close 40981 (time 1709788400.55 total 1563.39 last 24.56) - open/close 41233 (time 1709788410.59 total 1573.42 last 25.12) - open/close 41476 (time 1709788420.62 total 1583.45 last 24.23) - open/close 41702 (time 1709788430.66 total 1593.49 last 22.50) - open/close 41933 (time 1709788440.73 total 1603.57 last 22.93) - open/close 42149 (time 1709788450.75 total 1613.59 last 21.56) - open/close 42372 (time 1709788460.78 total 1623.62 last 22.23) - open/close 42596 (time 1709788470.80 total 1633.63 last 22.37) - open/close 42832 (time 1709788480.82 total 1643.66 last 23.54) - open/close 43062 (time 1709788490.85 total 1653.68 last 22.94) - open/close 43297 (time 1709788500.86 total 1663.70 last 23.47) - open/close 43535 (time 1709788510.86 total 1673.70 last 23.79) - open/close 43775 (time 1709788520.90 total 1683.74 last 23.90) - open/close 44011 (time 1709788530.91 total 1693.74 last 23.59) - open/close 44251 (time 1709788540.91 total 1703.75 last 24.00) - open/close 44493 (time 1709788550.92 total 1713.75 last 24.19) - open/close 44731 (time 1709788560.92 total 1723.75 last 23.79) - open/close 44975 (time 1709788570.93 total 1733.76 last 24.38) - open/close 45192 (time 1709788580.97 total 1743.80 last 21.61) - open/close 45407 (time 1709788590.99 total 1753.83 last 21.45) - open/close 45641 (time 1709788601.01 total 1763.84 last 23.37) - open/close 45873 (time 1709788611.01 total 1773.85 last 23.19) - open/close 46110 (time 1709788621.05 total 1783.88 last 23.62) - open/close 46353 (time 1709788631.08 total 1793.92 last 24.22) - open/close 46573 (time 1709788641.12 total 1803.95 last 21.92) - open/close 46793 (time 1709788651.17 total 1814.00 last 21.89) - open/close 47012 (time 1709788661.21 total 1824.04 last 21.81) - open/close 47232 (time 1709788671.21 total 1834.04 last 22.00) - open/close 47437 (time 1709788681.25 total 1844.09 last 20.42) - open/close 47646 (time 1709788691.27 total 1854.11 last 20.85) - open/close 47851 (time 1709788701.31 total 1864.14 last 20.43) - open/close 48071 (time 1709788711.34 total 1874.17 last 21.93) - open/close 48285 (time 1709788721.37 total 1884.20 last 21.34) - open/close 48502 (time 1709788731.39 total 1894.22 last 21.65) - open/close 48713 (time 1709788741.42 total 1904.26 last 21.03) - open/close 48906 (time 1709788751.42 total 1914.26 last 19.30) - open/close 49111 (time 1709788761.43 total 1924.26 last 20.49) - open/close 49303 (time 1709788771.45 total 1934.29 last 19.15) - open/close 49502 (time 1709788781.49 total 1944.32 last 19.83) - open/close 49714 (time 1709788791.53 total 1954.36 last 21.12) - open/close 49913 (time 1709788801.54 total 1964.37 last 19.89) total: 50000 open/close in 1968.91 seconds: 25.39 ops/second