Developing a real-time automated trading platform with Python

Video thumbnail (Frame 0) Video thumbnail (Frame 3263) Video thumbnail (Frame 15532) Video thumbnail (Frame 16716) Video thumbnail (Frame 17773)
Video in TIB AV-Portal: Developing a real-time automated trading platform with Python

Formal Metadata

Developing a real-time automated trading platform with Python
Title of Series
Part Number
Number of Parts
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Miguel Sánchez de León Peque - Developing a real-time automated trading platform with Python Nowadays Python is the perfect environment for developing a real-time automated trading tool. In this talk we will discuss the development of: a general-purpose multiagent-system module using Pyro and ZeroMQ; a platform, based on it, for developing automated trading strategies using Numpy, Numba, Theano, etc.; and a GUI for visualizing real-time market data using PyQtGraph and Qt. ----- In OpenSistemas we have developed a general-purpose multi-agent system which is written in pure Python: *osBrain*. Agents communicate with each other using ZeroMQ, allowing the user to define different communication patterns based on their needs. Based on this multi-agent system, we have also developed a broker- independent platform for real-time automated trading: *osMarkets*. This platform implements specialized agents: - **Feeder** is an agent which receives real-time data from the broker. - **Router** is an agent which receives data from feeders. It manages the historical data and distributes updates to all the subscribed agents in the network. - **Brain** is the most common agent. It receives data from router or from other brains and processes them, sending the results to other brains or sending orders to be executed. Brains can make use of many useful packages avilable in the Python ecosystem: NumPy, SciPy, Numba, Theano... - **Trader** is an agent which is designed to interact with the broker, just as the feeder, but to execute market orders. While it is still in its earliest stages, we are developing a tool for real-time visualization of trading strategies using PyQtGraph. This tool acts as an agent in the multi-agent system.
Meta element Game controller Presentation of a group Statistics Execution unit Source code Workstation <Musikinstrument> ACID Division (mathematics) Real-time operating system Computer programming Goodness of fit Machine learning Strategy game Computer configuration Different (Kate Ryan album) Internetworking Bridging (networking) Operating system Search engine (computing) Metropolitan area network Programming language Workstation <Musikinstrument> Software engineering Software developer Projective plane Shared memory Coma Berenices Division (mathematics) Price index Offenes Kommunikationssystem Computer programming Graphical user interface Integrated development environment Computing platform
Group action Transformation (function) Beta function State of matter View (database) Multiplication sign 1 (number) Real-time operating system Parameter (computer programming) Data analysis Function (mathematics) Computer programming Independence (probability theory) Order (biology) Machine learning Different (Kate Ryan album) Semiconductor memory Visualization (computer graphics) Pressure volume diagram Cuboid Electronic visual display Arrow of time Circle Series (mathematics) Abstraction Multiplication Physical system Algorithm Graphics library View (database) Real number Power series Moment (mathematics) Price index Offenes Kommunikationssystem Open set Thread (computing) Message passing Sample (statistics) Linker (computing) Series (mathematics) Architecture Order (biology) Buffer solution output Right angle Pattern language Energy level Video game console Quicksort Router (computing) Point (geometry) Algorithm Real number Branch (computer science) Rule of inference Portable communications device Metadata Emulation Wave packet Number Multi-agent system Hierarchy Operator (mathematics) Energy level Service-oriented architecture Data structure Associative property Computing platform Directed graph User interface Rule of inference Dialect Information Copyright infringement Interactive television Independence (probability theory) Planning Cartesian coordinate system Graphical user interface Particle system Spring (hydrology) Personal digital assistant Calculation Computing platform Electronic visual display Service-oriented architecture Abstraction Window Library (computing)
Frequency Graph (mathematics) Computer configuration Code Personal digital assistant Single-precision floating-point format Line (geometry) 2 (number)
Complex (psychology) Group action Concurrency (computer science) Serial port Logistic distribution Information overload Execution unit Source code Real-time operating system Client (computing) Open set Computer programming Neuroinformatik Embedded system Medical imaging Sign (mathematics) Array data structure Different (Kate Ryan album) Object (grammar) Local ring Error message Information security Physical system Social class Exception handling Mainframe computer Digitizing Software developer Keyboard shortcut Sampling (statistics) Database transaction Bit Unit testing 10 (number) Orbit Data stream Arithmetic mean Message passing Data management Process (computing) Series (mathematics) Telecommunication Order (biology) Pattern language Problemorientierte Programmiersprache Summierbarkeit Ranking Quicksort Arithmetic progression Aliasing Socket-Schnittstelle Open source Computer file Connectivity (graph theory) Similarity (geometry) Control flow Streaming media Event horizon Product (business) Number Architecture Frequency Crash (computing) Energy level Implementation Associative property Address space Form (programming) Default (computer science) Standard deviation Focus (optics) Matching (graph theory) Information Server (computing) Volume (thermodynamics) Cartesian coordinate system TDMA System call Graphical user interface Software Integrated development environment Personal digital assistant Network topology Social class Complex system Asymptotic analysis Digital electronics Transportation theory (mathematics) State of matter Code INTEGRAL Multiplication sign Decision theory 1 (number) Parameter (computer programming) Correlation and dependence Mereology Formal language Bit rate Cuboid Software framework Endliche Modelltheorie Position operator Stability theory Predictability Area Email Data storage device Multiple Regression Parallel port Price index Flow separation Type theory Architecture Configuration space Website output Right angle Remote procedure call Procedural programming Ocean current Classical physics Functional (mathematics) Server (computing) Implementation Link (knot theory) Divisor Virtual machine Menu (computing) Distance Graph coloring Twitter Power (physics) Wave packet Multi-agent system Root Object-oriented programming Internetworking Average Software testing Proxy server Multiplication Dialect Forcing (mathematics) Mathematical analysis Library catalog Symbol table Calculation Blog Object (grammar) Service-oriented architecture Local ring Abstraction
good morning everyone again I said like teachers to our the next speaker Miguel Sanchez didn't think was a software engineer at open system as a medium company based here in Madrid and he's going to be talking about domestic trading thank you solo welcome everybody to this presentation but unfold in real-time intimidatory I'm glad to see so many of you here today the following developed acid on this bridge has been developed in open system which is a company based in Madrid and in particular in their research and development department or division which is led by phenomena there about me as well uh man is because it's Olympic in and I'm modern industrial engineer with backgrounds inElectronics of motor control and Computer Science at a passion for programming data statistics and machine learning stuff Programming with Python about 2 years ago exactly when I started working with open systems as is being gradually falling in love with this programming language so what's the motivation behind this project if you go on the internet and search for currently available platforms for real-time of monitoring a unit and things like that station traitor indicator and that many many always each of them with their own little differences however all share some caveats so for example the oral probit sorry they are very heavily text of oriented so many of them cannot even right you don't have a fixed environment that the providing support what no and most of them they don't they do not provide non-Windows support and the most important questions perhaps can I use Python to create my strategies and the answer here is that the so what we wanted we wanted to have control over our own tools so we needed to have the source code to be able to modify that will and we wanted to have an optional graphical user interface we wanted this tool to be multi platform so that we were able to write in the listing the major operating systems and we wanted it to be program in the course so is
possible that the question but allow me to introduce you to always markets always markets a broker independent platform for real-time ultimately training with program dependent means it should be able to work with any broker available of there as long as cost broker provides you with a convenient API to interact with it is implemented over all s brain will see later what that means about for now I think it is sufficient know that always phrases which color multi-agent systems in which different agents or actors are independently and communicate with each other using message passing it is assigned for real-time automated trading 0 so that means it must be passed to this and it must be also robust this is another view of system architecture of always market we can see the input of the system is just the role market they are coming from the broker means that updates on market data or market prices and the output of the system is of course the buy or sell operations which are sent back to the broker will see what those little circles and there's our the agents sort the actors in the multi-agent system under the arrows in between groups and message passing names inside the circle so just the roles that the agents are playing because normally agents are specialized in a certain role that they play in the in in the overall system so we'll start describing this agents with different the the 1st 1 is called the beta of structuring the 1st one's called beta and its only purpose is to get market data from the broker that means that the state and cannot be independently from the from the fall apart from going to be independent so we must implement different there's 4 different brokers is also multithreaded agent was later what that means that this particular agent useful needs to deal at least with one-third that it's getting real time market updates and all that is able to attend requests from the agency in the in the system for for example historical they just that there on the right of the people we defined rule there were there is finally a brokering dependent provide this kind of like a theater but in this case is brokering the plan in its purpose is to store and distribute this market data among the agents in the system in all its markets so for the rest of the agents can subscribe to new updates that come from ruder it is also able to update the metadata that it has about 1 in memory we all the information that is coming from the theater and also is able to resample the that is necessary the market data is implemented using them pirates perhaps the main market data structures are that takes which are simply to asking and the prices at a given time stamp the bar which is usually the 1 and all them high-low-close prices during a time span and then and then the our series which is simply a series of fires sort of course by the time step as you may have noticed by a power series are updated in real time so in order to avoid full memory copies on each update what we do is we just use of view of a nonpareil base array or a buffer which is the 1 that we update with new bars so if a new particles which subdatabase array and then which simply change the view which is much faster than performing a full memory copy on the right of ruler we can find the brains and the brains are the most commonly called agents in the system and are able to subscribe to rooted in order to receive updates on market data and are able to subscribe also to all the branch of the sentence from the actual algorithms for ultimate training so they're and perhaps the most important agents in the system so they must be able also to send orders like buy or sell and the moment of course to the monitoring of other algorithms that would have implemented in an open system was that I think it's important to note that thanks to the great ecosystem available out there for data analysis and machine learning in by phone so you can do almost anything you want with his friends can make use of Mumbai you've on that I scikit-learn the on or whatever you want number so at the end you can't do anything that you can do with Python which usually means just anything 1 of the best things about this kind of a teacher is that you can create some kind of abstraction and in this way just creating a hierarchy in which the lowest level brains articles that are calculating using the real market data and then as we go up in the layers was start treating concepts more which are more abstract or more like how like human trade works so we can start talking about a market text intention market structure in order to consume kind of patterns and the last agents that we have here and the overviews the 1 on the right which is the traitor which is similar to the fear that means that it is broker dependent of course and its only purpose is to handle orders of because of bytes of Polish coming from the brains and then expects accuses the smallest against the broker so just to make it clear but here's an example imagine that we have heard of that system in which we don't have the will brain in there and then at some point with parameter we can do that in real time and then these brain requests on historical data through the there may not have this data available so we will send the request to fear it will get the data from the broker and then I will send it back to rule out from rooter back to bring all in all sinking and synchronous way but so from that point the brain may start sits subscribing to out real-time updates from the work of from the rooter and then performing some calculations in order to generate orders that will send a trader and also be made start publishing some information about the things that we have calculated within this spring and all the rest may need this information on right use may may be able to use it there are all the regions that we have mentioned such as a lover of lone pair or warning messages before which we can use to publish information about the accounts balance the other which is kind of a common console to control the agents that arise so we can kill some corresponding once we want we have also developed a system graphical user interface this graphical user interfaces for real-time market they have associations and also for real-time Association of indicators indicators are just can of mathematical transformation of the rule market data this graphical user interface is really doing well integrated with markets and was later how would you know how we you this from to implement this graphical user interface was used QT perhaps because it is the most widely used and also because it is really great it has really great portability but I think most importantly spike at a graph like that you have this much less known than QT so great tool which is written in pure python and it's a library for fast real time display and the best thing about this library is that it implements all the user interface of all the user interaction interaction story that we need to so finding or assuming 1 axis or to act sole implemented and working out of the box as well as updating in real time the axis so here's an overview all of the main window in which we can
see a list of and the currently running in France in the network and from the main window we can create new have certain windows with different and independent charts
apart from that we can see of
charges implemented this assault IQ graph OK we can change from here the data that we are displaying from markets or in this case we have just change the period of 1 2nd periods 2 seconds period and low we have the option to post automatic updates of you and then as you can see we can means a mild we see that everything works works pretty well and we haven't written single line of code for that so I could hear
and we also have this indicators menu in which we can select the indicators that we want to add to the charge we can have the must New Paltz as you can see what we can this indicators of indicators below the market and we can also add indicators are uploaded uh over the market for example is simple movement moving average and of course we are able to change the parameters of these indicators also in real time provided that the graphics you gets updated of course so you can remove indicators sound everything all that's assuming and having still works pretty well so far so it is
important OK all that uh this graphical user interfaces said was optional so it is a well integrated into always bring how did we do this simply because all the charts are agents in this multi-agent system OK so when we are visualized in this data is simply because the charge is subscribe to real-time updates from root and when we piecewise in those indicators it is also simply because we have subscribed to brain that is publishing this information to us so at the end all the calculations are performed in the multi-agent system of obtained always markets so I have explained earlier bid on how we have implemented the graphical user interface but I haven't explained how we implemented always markets or these multi-agent systems so this is always bring OK and it's the general populace and multi-agent systems written in Python and this job of multi-agent system implements independent agents the problem and communicate with each other using message-passing and in particular with always brain we have we were able to provide and basic configuration and deployment methods and was later how we do this let's say a general purpose and meaning that you can really use this for many of the things that you can use it for and not logistics logistics transportation or in the military and defense applications in our case we also use it for distributed computing so we use this agents that are distributed among different machines and then of course we distribute this some groups around selected back there and then each of the workers performer analysis and create a correlation matrix and calculate some kind of indicators and then we take into account to generate portfolios that are optimized so when I say that these agents are independent of the question is how why are they implemented fragile Processes offer as well you have met jails and you already know the answer is so of 1st processes digital knowledge you that it's better for you because you like would be a much simpler also said that we implemented a message passing how they would do this it was using 0 you there are many reasons the first one is just because it's a great product is so cool so really you you don't have a pretty good and it's it's it's of may 1 because it's really fun that to the recommended so right now we have an idea of what basic agencies uh we know that it is a system process crossing the only it implements methods binding on connecting with different patterns that this Spartans are implemented in ceramic you so we have the classic push-pull or Request-Reply a publish-subscribe model of it activates all incoming message of this mean use this means usually an agent is running on mainframe process which is just waiting for pulling on input sockets for a message to right and that's when we start executing some code in the agent's multi-setting with different may be used as well within the agent I know that to do so what we do is to create process sockets that's also serial implementation to communicate with the main threat and so we would we do always try to avoid concurrency we always use message passing even within the same agents i configuration I said it was pretty simple how we that bitch agents are independent but demand they must know all the addresses of all the agency we want to connect them OK so at the end there may be many many subjects perhaps create a configuration file would be to to to use would be too complex we're not interested on that and in fact for many agents for most of them assigning a random addresses perhaps simpler and more convenient we just don't care which others of these agencies binding to so if only we could just an area have something like a name server like an address book where we can reduce the orbits against them have something like a number of my and an entity that was able to know all the addresses that the agents are minded to another to send them to all the agents that are going to connect to the to the personal so is possible and here's where higher for comes in which is another great piece of software and also not much well known I think comes from python remote objects made allows us to treat remote it's a slope of this is done so with an object that is called a proxy and we can use this procedure call methods of that fact implemented in the remote regions and then these schools are serialized and sent to the remote agent executed there and then the return the serialized back to act so we can treat them just as if we were dealing with local objects but everything is being executed remotely so it's great very very convenient and apart from that already has a name server implementation so it's a win-win decision so now when we know what a real agent is a system processes in France of our multiplexers server on the server actually serves a nice an agent object but using parallel and then we have a main frame of I mentioned before the crimes may look this is what a school what we call in Paris so conclusion where we were able to implement a general purpose multi-agent systems with Python in which agents are independent and communicate with each other using message passing and we have achieved and it's a deployment of remote configuration process using pyrophoric so let's start with with the what some call samples just to understand how this works this is the most basic sample which is at war world and in this case as you can see think In here which were just running a name server around the MainServer underlining here were running an agent that we call example this is an alias and we tell them to reduce their this name server and what do we have here OK this variable is actually a proxy case it's not the remote of this function will start a new process and the system by the by forward bind to look holes and then with this uh with his this proxy we can call methods that are implemented in the remote so we can call for example the method like in which is just the information log and say hello world OK when we do this this call looking for a world of sterilized could sense it's been sent to the remote object remote agent and the Agent will execute the bucket is assessed 2nd example which is a simple 1 as well in which we're going to implement the push-pull butter OK so in this case we're doing justice before with creating a name server and then of the tree and into 2 agents sender and a receiver we call them involved and then of what we do is to combine the sender using the push button which is again this year and q communication patterns we give this circuit now it's OK but it is easier to refer to it later that way and then what we get we get in return is to address the actual address where the sender bind because we didn't specify a port for example so by default it binds to the local host and then they will select a random or because we just don't care so this call is executed in the remote region but then we get this variable here in this group OK and then what we do is again to serialize this call we're going to tell the receiver to connect to the status so this will be serialized and we're going to serialize the average which is a local optima and then we're going to see a light as well as a function of function that will be used as a handler the salaries implemented in here and it's red simplest just as before what we're going to do is to get this message method of that we received a simple lot some information and we will said received and the message so once we have done this we're going to start thinking it will and then we're going to start sending message from the center so what would what would we get in return became the consulate executed this we will start seeing messages saying received catalog received award receives at the focus will be on a large sample in this case In this case all always markets in this case we're going to be fine class OK that inherits from brain because we can also use object-oriented programming collective so we're just telling this state and that on you borrow this is the behavior so the the calls value right now is higher than before we're going to buy or what was going to so and then we send it off it's pretty simple and it's a stupid and do it with your money of course here what we're going to do is to create an on the picture this call this will simplify the creation of of all the common agents like the the there are all the root and the trader they will all be created and that we all they're working with on the work against the on the broker so we just have to provide a account information and then we're told the system to stop stream in real-time updates on Eurodollar there and then we can add a brain to the system which we specify the class so we realize again this class and this class that will be used to create an object in the remote agents in case and the brain and it would be served by the Arab and the state of then we can just call this subscribe method which will make this brain the brain example but we just created subscribe to our real-time updates from Roger on the Eurodollar error based on 1 minute period and that's it has a final conclusion about Python has been proved to be a great tool for implemented a real-time at military about and 1 last thing I'm really happy to communicate that always bread had just been released during the 1st the 1st and only outside of the the company so you can find this multi-agent system and and you cannot you want to check the source code of course you can find some documentation it's a work in progress because we don't have much time so so that city you want to contact us you can send in e-mail to me that you have my e-mail address for phenomenon you have also some links if you want to visit our website or blog you want to follow us on Twitter also we are hiring so if you like I thought you like training or you don't like any of them but you still like the analysis of big data or data association or whether integration or anything that you find in the world we do have a website and not hesitate to send us your your CD to architecture of the system out of so that's it thank you very much you have any questions from a certain distance thanks again for this great OK so we have the time for several questions and managers as to that we're regarding everything so please with forced to give you the microphone some high thank you for for the really interesting and test how those pyrophoric deal we are exceptions or areas like it's said traceback send back to their is silent in the face in that it seemed as tho it's not easy to of trees but like the the full respect the serialize it so now just that can get the message with the trace back so you can use the traceback thought form x from exceptional get that information that is useful for you but of course you usually have agents that work well because you have test that validate the functionality without the need to run it remotely writers you write tests locally validate the agent and then to run it remotely but anyway it's a tragedy it's to get some information OK but not everything approach that but I users by default well at least in always bring was brain uses by default the killed facilities and can use also deal if you are dealing with the prox is OK but it's still much better the the and and with this the performance of the so many transaction Canyon is doing a 2nd or a year because Python it's considered to be slow so as you see by Don these days standard that but it's not like just 5 from here we use about some because of for us there has been proved to be sufficiently fast for real time course that depends on your application our performance is limited to the performance of high you and Q which is in from the Indians force you and Q which is the message passing that issues between the agents I mean you and on viral are kind of slower but we only use that for deployment and configuration of that all message-passing is used this on use and pickle for sterilization and and Q. or PageRank you fast so that will be our bottleneck but anyway so bottlenecks always with international against the broker it's always was much slower to get messages from the program that to send messages to the broker through the internet like bite that could take out in all 50 ms but anyway for real time it's more than enough right so 1st clothing for much for people and I would have lots of questions namely our history including on this kind of problems were handled through which tools that are labeled as a C B of complex event processing and a lot of companies have invested a lot of money to solve that problem you have start from Twitter which was also open source you have to them a land of architecture from linking sums of Sparta a gazillion tools and I'm just interested in what did you feel that you have to invent something similar and which is like very similar to the existing tools that would be the 1st part of the question why may disagree with that they are very similar this is completely written in Python that may be an advantage it is an advantage to us and we're like that in this product of course has evolved OK we can have this intention within 1 to implement these at 1st right we started simply use insulin humans separate processes but at the end we find out that these tools could be glued together and worked pretty well so so for example the use of I don't think you can do things like that with those tools that your mentioning and also well having a full Python implementation and yeah that's a great answer the problems mentioned are of course solved in a different way in different frameworks but OK in fast so the other part would be on I didn't know what it is that you want to talk about the edge cases which are like very important in on frameworks like this and especially if you're playing with money so I'm talking about things like on how old are you guaranteeing that the events are processed exactly once and so on we know that if you are connected to 1 or several data streams How are you really sure that's the events that are coming in our really order because it could happen that you get from 1 summary array what sorry I really or do however like looking 1 after another because it could happen that you get something from 1 of its time t but because of of latency issues or whatever you get from In the best of or whatever exchange fundamental which are happened before that but later that additionally people use something like color which slows you're looking problems in distributed systems and because of course traditional you have a lot of smaller systems which wants a which wants to x is the latest state of the system so you have to lock in problems that was a long question anyway yeah I thank you and the other thing is that which pursuant you and that's what I like about this architecture you're free to implement Europe message-passing so you mean how can you be sure that you all received messages 1 well it depends on your application is that it's important for you you can implement manually both patterns for communications for our case in fact which is just publish-subscribe because in real time we don't care even if we don't receive 1 take ones which is something that doesn't happen but we don't care which is half of all of the year and we want to process it as fast as possible if there is little wrong data or some missing data we don't care we just want to keep working working normally that would that 1 affect your strategy so you're not afraid to that you would make up on based on different data the same decision twice because you're workers would process the same and twice 4 3 times and you would then basically go with the same box as much capital or something like that in European question short so you're saying that you're not afraid that's based on 1 went are the workers would processes through process into multiple times and you know that that won't happen was sort you understood that not that will happen because we are listening send in those that that they at once look at the worst that can happen is that we need a packet can message once the we care about that because usually use have fatigue so maybe your bar would be formed a little bit different have lived with a little bit different but that's not really important factor when you change brokers the data that you get from those brokers is also different so it's not important if the the energy gap is a 1 per cent difference which is which is not this week we don't care OK so that's a good thing about it is that we can implement a flexible Mulberry Street message-passing such as publish that works pretty well on pretty fast we are more interested in speed time in for example getting all the exact date of brokers is provided us because we don't trust data and we not that much the key it's 1st all thank you for this wonderful book and the question is that in such a multi-agent systems so that we can do a lot of problem users for example that's 1 of the processes of language and I am expecting that and so hold to someone all this overall system only 0 0 what is the use the reaction if for example some agent dies yeah think Cost of monitoring is also implemented by the power the book so that depends on your application we have we are monitoring our against and brains and 1 of the best things about this architecture is that exactly 1 brain for example that crashes which is something that should never happen and never happens since then you can kill with and you can respond it and everything works as expected and why is that and that again thanks to the rescue because for example you can find which started and connect to our to an address that is not find did so there's no problem if there's a client that is connected or subscribe to on of brain that publishing information than this rank and I we can kill with we can create a new 1 and it will start receiving this information is it 1 break anything around so it's not completely independent thank you and I'm it's going to look at NumPy and race and of why you why did you choose those what features are we looking for and that you can find in in basic types there's this woman and segment agree with that and the last 2 uh questions about so such complex system you could have an agent going down coming up of the discontinuity and that worries me greatly about especially refinements and sorry can you repeat the question can you speak so the smaller staring at the at the question was why numpy arrays object and also to uh uh reassure me about discontinuities nodes popping up up and down and you talk about pickles and until it's not because of the internet is is scary yeah and also that there is something about OK answered 2nd 1 1st sentence of about the guilt over the internet is to is this system is completely unsafe by default so it is meant to be executed in a controlled or restricted environment for example in cluster or or just in your own computer if you want to communicate between agents in remote machines and you have to pass over the internet then it is up to you to implement it the security so you may want to use that began around SSH tunnel but of course this is not secure it's always security fewer programming and for the 1st question why number arrays of the answer is that all make a system available out there for the analysis but all that most of the existing seems to work well with the number arrays so why would why wouldn't we use NumPy arrays OK will and why would need because the more expensive and I was wondering what features you're after but don't worry about it and that the the idea that a brain that doesn't use your own DSL black trees just pure Python code I don't see what you put itself is besides a collection of already existing frameworks for creating a large systems and the and and struggling to see whether it can you ask again regression models or have difficult for sorry on you and I don't have a DSL a domain-specific language which abstracts away all the bad things that you could do as a developer when writing a brain power so what does your product offering tens of stability unpredictability the components will work together as a system because I don't see any of its knowledge reduce just implement test of the independent agencies and they're not the whole ecosystem and that's it I'm sorry about all of the questions you you get kind of kind where you are not using these from that the quality of my question to ask about the image this is about the spirit of the great induced by Don and I would expect to be would be in this kind of a situation and you are not using data yes yeah these are things like so as as a said this has been evolving in the in the past months so we didn't start with the idea of creating our own tools but that just happened I mean we liked to Cuba match and all the tools are your mentioning implemented this flexible low level of communication patterns or not and also I don't know if they implement some things like our ideas we can implement tools so using that that framework for example in the news in parallel as well for the computerization yeah maybe maybe that works as well binary and so on I'm very interested in how are you thinking or hard of approaching testing this framework because as you already mentioned it's a compositional separate agents and so on during are will broke approaching the testing and on how are you testing of the whole system just using and just what kind of test harder and it was you could recommend so what is your experience that because I think it's varies a D N a very different problem then what we are going to have to most of us here probably from each day I mean we know we are aware of the tests for the web sites and things like that and this kind of problem looks a bit different so I would be interested in how are you test is all we're just in this unit using unit tests and pi test on that's it the new way out of the habit to talk to you and I'm sure you will supply better tools for doing this perhaps a and 1 quick question in in finance trading due to your brain must know the current states of for example volume of open positions and I would like to know how you save the state and value replace what needed to know the state and if for example of a brain dies so value the positions both European positions are usually was stored in the broker of course was stored locally as well that depends on your implementation you can make traitor store that information or you can make it if you're going to propose to implement some kind of we are handling of the or the implemented in the brain and we would go out the same mentioned I think before or other agents such as former which are publishing information about for example the account stage or the rates are which you can trade your account gradually against other currencies so in this in this sense system with a flexible minutes it's up to you great thank you to all of you for asking so many interesting questions and the signing thanking all the speakers from this morning