Lessons learnt building a medical chatbot in Python

Video thumbnail (Frame 0) Video thumbnail (Frame 5509) Video thumbnail (Frame 7367) Video thumbnail (Frame 8273) Video thumbnail (Frame 9540) Video thumbnail (Frame 10425) Video thumbnail (Frame 11995) Video thumbnail (Frame 13497) Video thumbnail (Frame 17122) Video thumbnail (Frame 20201) Video thumbnail (Frame 24179) Video thumbnail (Frame 25162) Video thumbnail (Frame 27380) Video thumbnail (Frame 29052) Video thumbnail (Frame 33510) Video thumbnail (Frame 34498) Video thumbnail (Frame 36020) Video thumbnail (Frame 37409) Video thumbnail (Frame 41186) Video thumbnail (Frame 51101) Video thumbnail (Frame 52266) Video thumbnail (Frame 53618) Video thumbnail (Frame 54832) Video thumbnail (Frame 56356) Video thumbnail (Frame 57210) Video thumbnail (Frame 63368) Video thumbnail (Frame 64931)
Video in TIB AV-Portal: Lessons learnt building a medical chatbot in Python

Formal Metadata

Lessons learnt building a medical chatbot in Python
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Lessons learnt building a medical chatbot in Python [EuroPython 2017 - Talk - 2017-07-14 - Anfiteatro 2] [Rimini, Italy] ""To put an accessible and affordable health service in the hands of every person on earth."" Thats our mission at babylon. Leveraging the powers of AI, python and micro services we took a step towards that vision by building a medical chatbot that we shipped in November last year. In this talk I would like to share with you all the things we learnt in the process. This talk is our story. Its a story that starts with an idea and meanders through the dark and dangerous land of things like Graph databases, machine learning and async programming in python. The story is far from over, but we have come to a point where we would like to reflect and share with the community all that we have learnt. More specifically I will cover: - Architecture decisions we made and why we made them - lessons learnt doing async in python at scale - testing chatbots - clinical governance and safety (literally 2 sentences, I promise) - The drawbacks of REST - Why I am glad we did most of it in Python And then of course some time for questions at the end :
Chatterbot Intel Service (economics) Information Concentric Multiplication sign Cloud computing Mereology Disk read-and-write head Cartesian coordinate system Information technology consulting Number Wave packet Degree (graph theory) Goodness of fit Process (computing) Internetworking Term (mathematics) Software Videoconferencing Video game Smartphone Arithmetic progression Telemedizin
Intel Scaling (geometry) Multiplication sign Demo (music) Virtual machine Coma Berenices Lattice (order) Shape (magazine) Neuroinformatik Software Software Right angle Quicksort Mathematical optimization Directed graph
Scaling (geometry) Mapping Demo (music) Multiplication sign Table (information)
Sign (mathematics) Computer chess Direction (geometry) Multiplication sign Maxima and minima Planning Stress (mechanics) Limit (category theory) Bit rate Measurement
Chatterbot Software bug Building Natural number Interface (computing) Sheaf (mathematics) Client (computing) Formal language Software bug
Query language Network topology Decision theory Real number Execution unit Combinational logic Set (mathematics) Branch (computer science) Figurate number Number Physical system Software bug
Revision control Complex (psychology) Software bug Mathematics Graph (mathematics) Logic Decision theory Network topology Projective plane Canadian Mathematical Society Database Sphere
Building Content management system Graph (mathematics) Decision theory Multiplication sign Direction (geometry) Branch (computer science) Semantics (computer science) Software bug Twitter Goodness of fit Latent heat Hierarchy Cuboid Endliche Modelltheorie Software bug Graph (mathematics) Matching (graph theory) Mapping Information Relational database Database Semantics (computer science) Hierarchy Data model Software Personal digital assistant Network topology Bijection Quicksort Resultant
Group action Graph (mathematics) Theory of relativity Sequel Relational database INTEGRAL Multiplication sign Database Mereology Food energy Process modeling Software bug Data model Data model Process (computing) Hash function Robotics Social class
Multiplication Graph (mathematics) Multiplication sign Structural load Mikroblog Database Element (mathematics) Software bug Medical imaging Mathematics Message passing Bit rate Operator (mathematics) Green's function Chain Data conversion Endliche Modelltheorie Spacetime
Graph (mathematics) Real number Database Grass (card game) Proper map Product (business) Element (mathematics) Human migration Frequency Particle system Data model Mathematics Query language Chain Linearization Data conversion Endliche Modelltheorie Determinant Resultant Tuple
Human migration Constraint (mathematics) Strategy game Relational database Query language Execution unit Database Grass (card game) Tunis
Service (economics) Virtual machine Set (mathematics) Mass Disk read-and-write head Newsletter Rule of inference Semantics (computer science) Product (business) Software bug Machine learning Different (Kate Ryan album) Physical system Window Chatterbot Software bug Pattern recognition Mapping Building Content (media) Infinity Computer network Digital signal Formal language Demoscene Entire function Sign (mathematics) Event horizon Query language Arrow of time Video game output Natural language Library (computing) Spacetime
Query language Single-precision floating-point format Multiplication sign Virtual machine Cartesian coordinate system Directed graph
Point (geometry) Addition Computer font Building Multiplication Mapping State of matter Multiplication sign Virtual machine Combinational logic Content (media) 1 (number) Insertion loss Dimensional analysis Symbol table Word Process (computing) Different (Kate Ryan album) Query language output Right angle Endliche Modelltheorie
Standard deviation Complex (psychology) Web crawler Scripting language Code Multiplication sign Combinational logic 1 (number) Set (mathematics) Client (computing) Mereology Formal language Data model Online chat Mathematics Bit rate Different (Kate Ryan album) Endliche Modelltheorie Data conversion Office suite Physical system Scripting language Pattern recognition Mapping Structural load Moment (mathematics) Sampling (statistics) Infinity Bit Staff (military) Virtual machine Process (computing) output Website Software testing Bounded variation Writing Spacetime Wide area network Point (geometry) Game controller Service (economics) Computer file Connectivity (graph theory) Virtual machine Similarity (geometry) Online help Mass Rule of inference Product (business) Wave packet Element (mathematics) Number Operator (mathematics) Authorization Software testing Traffic reporting Mathematical optimization Mobile Web Scale (map) Standard deviation Scaling (geometry) Validity (statistics) Cellular automaton Content (media) Interactive television Deep Web Cartesian coordinate system Word Query language Search engine (computing) Network topology Information retrieval Natural language
Mobile Web Type theory Array data structure Macro (computer science) Validity (statistics) Code String (computer science) Connectivity (graph theory) Object (grammar) Client (computing) Musical ensemble Computer programming
Matching (graph theory) Cone penetration test Root Multiplication sign Electronic mailing list Mass
Axiom of choice Electric generator Combinational logic Mass Frame problem Web 2.0 Mathematics Workload CAN bus Internetworking Cuboid Software framework Physical system
Workload Chemical equation Video game Error message Directed graph
Wechselseitige Information Randomization Context awareness Functional (mathematics) Building Module (mathematics) Service (economics) Line (geometry) Multiplication sign Characteristic polynomial Survival analysis Mereology Protein Generating function Event horizon Leak Frequency Latent heat Read-only memory Semiconductor memory Personal digital assistant Software framework Error message Predictability Beat (acoustics) Rule of inference Dependent and independent variables Electric generator Graph (mathematics) Basis <Mathematik> SchlĂĽsselverteilung Message passing Arithmetic mean Data management Event horizon Prediction Statement (computer science) Video game Natural language Iteration Figurate number Remote procedure call Object (grammar) Spacetime Vacuum
Context awareness Moment (mathematics)
Complex (psychology) Gender Figurate number
and so I went ahead and I work at a public health and and and material provision to be years and you very much for having so to
provide an affordable and accessible provide affordable and accessible health care to everyone and that is the mission of the company I work for and and it's been a real privilege of the last 2 years to Spain's the majority of my waking life trying to solve this problem and and when I joined 2 years ago we were really small company they were like yeah 40 of us and we really kind of and then on the accessibility part of all of this question of the 2nd statement because at the time and if you a safe you somehow have to physically transport yours according to some like health at the center where is a high concentration of salt the professionals that can help you with your illness and in an age where everything from information to capture videos was highly highly accessible it just seemed well within our reach that uh we can make health a significant uh to degree more accessible by leveraging the Internet smartphones and modern technology so and we went for its ends in July 2015 uh Babylon launched their answer to this accessibility question with their telemedicine applications that essentially allow people to and and have to consultation with a doctor over the following and and the whole kind of health care service that goes around that so after the team kind of recovered from literally doing like blistering hotfixes while our CEO is stepping up onto the stage at the Royal Albert Hall it will relax that themselves on the back for making health care a significant degree uh more accessible but the affordability parts it is the tricky part because 2 thirds of the costs of health care going to and the salaries of doctors nurses hotter professionals such so the so we need to figure out how do we solve this problem how hot it if we want to make healthcare more affordable we have to somehow reduce the cost of health care professionals so 1 way we can do this is by simply just increasing the number of doctors and and the thing that kind of a reasonable approach but some people estimated we or 5 million Dr. short from providing and affordable and appropriate complicated when you and not just that doctors are expensive to train they can take a long time to train iterated very difficult training process so this approach is using the local press them in the short term so affordability is really tricky recover made good progress in accessibility and I would have my head down text like that added features and kind of went on our merry way but then 1 day highly policy over watch the movie and
the so if we do now
out who really and ever since that fateful day
the this question came up again and again and again in meeting after meeting after meeting as this kind of idea was starting to take shape and in his mind and they say things 20 times of repetition before something source sink and and and optimization kind of relentless repetition this idea other 2 and take shape in the in the in the imaginations of the engineers all and then the the idea is basically quite simple right so we know we have lots and lots of doctors and short in the world and and doctors legitimate inherently difficult scale but we know something that's easy to scale we know that machines and software and computers are are really no trivial to scale so if you could just figure out a way to kind of put some all of the doctors brain in sort of a computer just 1 computer then we can
just scale it really easy and then iterate and that so this is kind of like that you know is a really good daunting you know idea and you know how to do this medicine is so complex to before we did before we did anything we build a demo we want to see how people would take to this idea how people would feel like sharing you know they have the concerns and with the maps table the
demo the time of the
and yes and the
and he
the the so always in the Irish team
at the time interval Desdemona and and uh while I was holding the market saying to ILI like Iliana hardenable this like this stuff that written is suggesting we can do in this game of like a but no idea how we are going to do this but the reception of this lowest so you have an amazing people a lot of people with certain kind of your people thinking in this direction and the implications it has for health care around the world that was just that they must have enough motivation and inspiration to not knowing how we wouldn't do it just kind of stay plan and see what we can do we so on that take is to a bunch of things we we
we we did a new challenges
we had at the endowment of about the role that Python play this whole journey and at the end of each section I'm going to you and to share with you some of the lessons we learned the something the we did like all works surprisingly well and lot of things we did we would like wow that was a really bad idea so let's start with go triage and for those who don't know what which realizes military origin is essentially and
you're right somewhere with a set of symptoms and someone figures out what you should do so should you go to the hospital with the pharmacy stay at home and you know monitor for a couple of hours like not kind of the the stapler diagnosis for this kind of suggesting what you should do so we knew we had to do this uh 1 that we knew we had to be able to technical queries and to suggest people what they should do next
and then building this medical triage system was a real roller coaster ride because we started all uh just being like all units is the basic decision tree you know we discuss the goal of this tree you different doctors to help a lot helpless and in which is kind of walk down the tree until we get with this number questions to ask and we can go and you give them the outcome at the end of that branch but then it's kind of like we you know we did the roller coaster of the quality of this question is way more important than this question and when these 2 questions up on some combination they can nullify each
other in our mind just blowing up the kind complexity of this kind of medical sphere and then we can apply come up again because we had some really good data decided helped us kind of like you know broke ice in these these problems and and uh and we both at the 1st uh because essentially what I can of going into the details of how we wait questions and that kind of stuff and it is used as a decision
tree that you kind of walked out with some kind of intermediate logic in between and we we both initial version in PHP with must equal and it just wasn't working for us like those clunky it was difficult to change and some the changes of quite risky so we we scrap the whole project can we rebuild it with Python and a graph database so this is a picture of the and the CMS that we used for a SCOP database and it's a small town of
corner of this this very very large uh decision tree the so why did we choose a graph database for triage I would love to say that there was no height involved in all decision but the truth is that at the time got databases are all the rage like even now a lot of hype about it and and and solving interesting problems and we're looking to get this
technology but they often good reasons the 3 and the graph databases is applicable for this use case so in this decision tree there's a massive a sense of hierarchy is branches of branches of the sister and this kind of uh a data model is normally a goods you know probably that may be a GOP database will be applicable the next thing is that the got database has all the semantics we need to reason about the decision tree it is out of box and I get the amazed that I showed you we didn't kind of build that do we ourselves like those a like crop semantics we have nodes relationships interrelationships have directions all these things that we have not been a model so reasoning about we wouldn't have to kind of gold is a mapping from the relational database to our kind of you know and this decision tree and and because the semantics kind of mostly match one-to-one and stuff like that some the tooling that comes up the box you know really was really really helpful without a lot of work and also assume as being and an example of that so what are some lessons learned and the 1st thing I would say this comes true for all software technology is a new technology is trying to try get carried away by the hierarchical these new technologies come out a lot of attention which means result of Twitter exigir and it and full and informed decision on behalf of the company it can it can in really bad the 2nd thing that I want to say is that this is a specific graph databases is don't get carried away by cool animations who technology that can sell its technologies so while sort of like really cool animation that
looks like at solving problems and you can't even imagine solving in your from a traditional way of of work so when should you use got databases now the single most important question
that you need to answer it when considering the with energy is a graph databases the following are relationships the most important part of my data model because if the answer to this question is maybe or sometimes for I suppose you probably get that had a bad time because and if relationships are the most important thing you of a data model there's just so much that you can do with a graph database now the reason this question is important is because In traditional databases about like relational databases and relationships are represented or or our and handled was joins and joins can be expensive and I use candidate tentatively because uh this is many kinds of joins hash join the merge joins in niche joined in other joins it up toward the robot and database that there's a lot of work to optimize these processes but to get the most out of it you often points of brought in a very specific way that makes your data model kind of quite a brutal and integration of the database quite brutal with graph database because relation with the 1st class citizens it as long as that remains true you can have a wonderful time and in graph databases and the 2nd lesson we learned is that data modeling is extremely important and this is this is very true and no sequel databases but especially true in graph databases and then we unfortunately learned this the hard way so you we did a medical triage with a graph database the group because that would
so well we also decided to use and you've got a graph database for all and chat what that model so what you can see here and there is a typical thing that you'll find the graph database so the rate noted is the use of the green nodes or conversations and blue nodes are elements and inside the conversation and element being basically everything that's in the conversation that is
mostly messages between users and as well as like conversation status updates and stuff like that ignore the purple notable just call those have nodes for known and but was shown this to the 1st in this room with this the concierge from from the image but the the node most closest to you the conversation it is the most recent elements in the conversation and at the end of the chain is the 1 for this and further from the past the reason why this is that every time we add new elements to the conversation with multiple of the operations happening we have to delete the 1 relationship change the status of the other relationship and recreate another relationship and which is a really heavy in error-prone operation that happens and I'm not only that it it exposed some no bugs in new for J and which at the time was that the fixed this but at the time when he deleted relationships between clearer at this space because you have been deleting relationships like crazy eventually our our and the spaces but used up and so what else is wrong with this and or seek on really see it here but so the red nodes are users and uh initially would want to represent the Babylon user like the other checkpoint as a user was appalled like creates so you know it will become to what initially they'll be chatting to Babylon about just what user and then some time but the doctor will come into the conversation and you know but what this means is that because most users talking to Babylon that meant that the Babylon using nodes had a relationship to every single conversation and what this means is that every conversation load and higher graph database was only 1 node away from every other
conversations and and graph databases called a supernova and supernode is like a really bad it's really bad performance in the budget ways you can do to get around this but none of it feels good and you reached on good on that period and all these kind of things we learned like in production and stuff like that which you really don't do and real way way more hits and now and about that adopting new technologies and making sure we do kind of proper investigation before we dive into something so now the reason why data modeling is important is because in graph
databases uh migrations on non-trivial by a long shot so what you see here is uh this should be 1 linear chain of elements in a conversation in just you this that runs a disease that this should be 1 thing but what you can see here that this is craziness happening like like those responding with themselves and stuff like that um and this is a result of of cypher query someone committed a change to 1 of the queries and just messed up the way that we work on adding new elements the conversation and it was causing problems for use in production so we had to fix it now write to him at a migration strip that fixes is not only because it's in determinants and this is really tricky so you really want a huge upon attention to your data models of oneself in this kind of situation the so the the thing we learned there is an basically the grass is always greener on the other side so we know how to make a geologist perfect forgotten databases but uh chaff what not so much so we're busy migrating the chapel that model over tuples Kras and and kind of coming back to this place you kind of feel synonymous to the particle 110
because we feel that we could have you know been of at new for j land promiscuous in both places has always been solid for us you know has than anything wrong
and returning now and just opening in a region has open on so stuff that we usually you never felt restrictive now units so amazing stuff like the constraints on your columns they disliked node blowing online suffer familiar querying and strategies and uh no charting that is easy is easy to do in relational databases database migrations are like amazing all the tuning that comes around it like a basic welcome you know and and and just the general tooling around uh which creates an relational databases in Python is just so great to come back to all that stuff and so here's the the grass isn't always greener on the other side OK so we have medical triage brilliant and we launched its it did really really well and we we
replace that against real life aiming this isn't doctors and adjusted really really
well we were really products and and you know we can approximate the sense of achievement but as highly reminded us that we were still a long long way away from where you want to be so we knew we had to bow check but and the we have since found out that building a chat what is really difficult to
and the reason is really difficult is because natural language is really difficult natural language is like what the problem that you're you approach the problem and you just imagine is going to be hard you have a subunit this again you really difficult but as soon as your skates touched the Iights you realize that totally underestimated how difficult it's going to be and the reason is difficult is because the scope of its just so huge and you interested start solving 1 problem and you realize what this means this and this means that the next minute you have given to try to put the entire kind of semantics and linguistic rules of of and all the things that happens behind the behind the scenes in your head and trying to solve that and the biggest problem we had to overcome initially was filling out how we could reduce the scope all the natural language understanding is something that we could take a bite at and should productions of our users so what we have to do it so that the fertilization that to make is that we are not interested in helping you decide whether or not he man but that 1 can help with the problem we want a home in on uh focus helping me with many queries so that's a very over here we have a set of services so we have this very warped performing medical triage we have a bunch of content you know content about different diseases and as massive library of content we also have uh doctors who are ready to be all machine content new query to handle your queries so we have this very can of contain the scope of the year and we have the services of here so when we put it like that all of a sudden it becomes a machine learning problem because we have an infinite space of inputs and then we have this kind of sick of defined outcomes and this is kind of cold into it intent recognition we wanted to take the inputs and map it to sum up what that we could do if the mapping was not good some generic fall-back that would have redirected to it offers medical queries it so it's a machine learning problem then we need data and make data isn't easy to find but we were really lucky because we already had this system running in production cold lost them in on
application we descend as like a single MIT's Mr. doctor and you would apply in a couple of hours with something like medical about what you should you and and the doctors and nurses who were doing this had the good sense to tag each question and
outcome so we have thousands and thousands of these queries it's about time that we realised that wanted to do this we had this really really rich data source and you know really for us to to train on so rich data source so you know some machine learning scientists and engineers that's all you need right not quite
because you know what the inputs of coming and was still free takes and free text is just inherently unstructured so we had to figure out a way to to take free text and then represented in a way that all models could understand they'll machine learning guys that a bunch of words and build the building multiple different classifiers to transform the the takes into slightly different combinations which is with the bag I would too big things other things were like the medical uh in extracted concepts possible to models and at the same time that some classifiers around our content in our pockets to to make it easy to map so essentially what they were trying to do is take a sentence and then put it somewhat a matter and and then the hope is that sentences that
symbol kind of cluster together in this methods reasoner to know a point on a map on some kind of to be met but the difference is that in these maps with the 108 and 80 dimensions so this is like you know it's a simplified the of states with some this data and what you can see is that in the top layer those red ones and are all about you make something like that on the right we have someone's by hand and foot and stuff forget and in the middle of those queries walleye uh one's about your nation and proving loss would the other note that political great great thing that is and but you know it's unclear job models like this it's about the addition or is this about her abdomen and or the reproduction
or or anything to this some parts that are quite varied but they all you can see a similar things cluster really well together and and the guy's going iterated on the models in in it go to a place that was doing really really well it was handling operations 95 % accuracy and 99 . 8 % safety all explain but more about what this what this is saying to me in in a bit and they use a bunch of technology a bunch of different technologies and that are available to you when you when you couldn't Python uh the user can suppose while they use a combination of the origins and other deep nets and I got to a place where you really how happy with their into the re recognition classifiers such we were doing really well and we're taking somebody's medical queries and we just not not not optimal handling and so so well but then you get a reasonable query and we just totally bomb with it would look so so stupid and then I would get another query that we just had an obligation not to mess but we don't really have good data for these queries so if the engineer is uh told the machine learning guys to to pipe down that had been at the time which can reject this problem so we didn't quite do that we use this tool called drive script um which essentially is this linguistic via cell that help you can construct linguistic rules to catch the variations of sentences and that interspecific outcome so this is an example of this is a query about uh looks like some kind of use with which all the reasons that um and it had a huge variation of sentences in this 1 rule and maps to a out of that we want to provide users so we we we we wrote like hundreds and hundreds of of these rules to handle those like non-medical but kind of important queries I mean were were were busy actually moving away from that of the the machine learning doesn't have had a chance to both things are more generic than the United States and can handle simple queries better than these rules can be so would be like we learned the testing chat what's a really difficult since since doing this have become really interested in how all
search engines test search engine because users have an infinite amount of inputs and you know like a website we have a bunch of buttons and then it near a certain amount of combinations you know of light and kept the buttons and stuff and you can write brains of things that make you really sure that your system is working but if you have an infinite input space you can make a change and not sure like what the change has done to some corner of this massive massive input space so we can come up with like a really good way to test the unity that give us cues short and so we just went for like many many layers of testing so the 1st thing we did was machine the the the model validations and which essentially is just uh in a putting some data side as it as a test training a model your your training set and then passing the trace it back to the model to see how all elements during that's where the 95 % accuracy come from and within right so I just thousands of arrive examples of examples suggest that uh linguistic use thing to is that many many many of these of these of these things and we also been created this kind of like uh clients mock system that that that pertains to client and uh with can be approach just just take loads of the interactions throughout the its and or be before making making major releases with the clinical validation tests where we would get a big number of doctors into the office and they will give you the chance what's going through examples and uh generate this report of connected Seyfert and move the post-market surveillance so when that when the products out there we have and doctors who build their main job is to take a sample each day and run for that example if they were having that query and and in the south and with a lot this was safe or not and that Soviet that 99 . 8 % safety from the 2nd lesson we learned and what is the chance what is only as good as a content so we initially thought measuring was going to be really difficult and is everything she usually effort into kind of solving that problem but in 1 of this kind of you know at the upper parts so that coming out we realize that this doesn't feel good you know or or natural language understanding is really good and they if feel good because our content wasn't like optimize we didn't spend much effort in our content and we've learned sense that you need to kind of improve both at the same time and it was good content opted understanding is really bad good understanding what the content is really bad as well to an optimal and improving the ML stuff for also improve the contents of the whole chapter because they're all together and then they asked machine learning guys like you know they must also what what what can I and can tell these guys that you have learned that doesn't do away with the current cool things you're working on that is spoken of and IP and control and they they said to me that basically a standard classification doesn't scale in the way that we've done here and distinct and they told me to use is that information retrieval is better than classification and the reason classification of the standard because someone make a small change that to retrain well model sometimes the the new feature we want to add doesn't quite fit with that what with that of the way because of these models and it is the Vericom slow process will be a hit you able to its rates are typically faster so we have obtained better language and staff and we have the maple tree or was we were doing really good but we had no way to kind of communicate mobile clients mandatory complex stuff happening in the back and wouldn't have liked this uniform way to communicate to our mobile clients so we had to do the dispatcher is where this like a simplified the this is more services are on this but and the dispatchers with that that understanding happens we have leaflets and medical entity extraction services that it talks to and added to the roster DSL service and but we needed to build an API that kind of simplified all this complexity happening in the back into a way that was consistent for all clients of mobile clients understand and and again this is just a massively simplified we have like that that 35 services going production and but it's this kind of peace that was particularly difficult to kind of take was complexity in that and a simple way for applies that and 1 of the lessons we learned is that uh just in this application which is an evil word because you start off like soluble this part it's just going to be easy where it's just you know request-response uh indices H P is perfect but it wasn't quite like that because that use which of those wasn't just let so that also can of bring a penis and seemingly and issue we likewise as you know 1 person to another person is just that these the people in his doctors authorization of elements and you know I just exploded with how much more complex this was than we thought it was going to be in a really useful tool we use that you help us in gold and client will the same time was connection which basically is is wrapper around cross that helps you and generates like documentation that your code is dependent on which means that your spider documentation is always up to date so how do words is basically the bolts like it's suspected him of Hollywood analysts like is like is like a standard for uh and how it got that and or it can design and so standard way of representing what your API does and you can do it and Jason or yeah moment so you go just like a file connection then Womack and the end point and that's like a file to the entry point in your code um and then it'll do lots of cool stuff for you so this is an example of conversation model so you can see there there's
some strings and pools and arrays referencing to other objects and in connection will do that type of a validation for you that you have to do in your code and it also means because of its connection is doing that you're really tied to making sure that this documentation is great because is going to impact the way that your programs which meant that our documentation was always very very good which is very useful to a mobile clients and and what you get then
is documentation and not just a convention the dates and API clients that during your mobile clients music taste not directly on thank
says cone so the last
thing I will talk about this just the role that Python played kind of an all of and I can really confidently say that we wouldn't have done me it wouldn't wouldn't then as much as we did in the time we did it with art and something like python and some the lessons learned were like root root not surprising but were like in a really welcome and and resoundingly true and some of this work what quite difficult and not ideal so 1 lists learned is that mass and Python are like a match made in heaven they are just so many amazing amazing tools that take in a really complex things and make them in a really easy but easy but you make them really accessible and and in this huge community run as well and is loads of
tools and stuff on the internet and tutorials that's kind of pitch on board see if you have like mass hey the uh workloads Python is that it's an excellent choice for you and not only that Python and the way it is also this is really the combination because again you have all these amazing tools like in everything from you know everything not the box frameworks like generative or simple marker frame of high false can of everything in between you have like native support to all modern uh web technologies like Apache Spark and and also but that you know they they strive is worst let's is is really really solid which makes building the stuff you know you re confident about police these complex
systems so the that the land and is that it if you have a
sink he the workload in Python you're probably the you know the court out at some stage
even if you know what you're doing is public enemy something is in a catcher out and because of all the to the specific knowledge you know about how thing works in Python so this is than screenshot from shock um and why we during packet sniffing you might ask no 1 wants to do this but we had this issue where like 1 in a thousand we doing this is sold error that's on balance and so as to strengthen it's a subset in and it is shown is with different each time and for the life of us can figure out what is causing this problem so we then went into the packets to figure out what is happening and
basically what's happening here is the service and like I have a message to the this over the we this concept to think it key exchange and size uh and stick and handshakes and stuff and Iraq amid only the Cervantes's hello again and the receiving survival of all expecting that like failed and we couldn't figure out why this like a random part of initial Hallowe'en in it's all handshake which is kind of getting and there and we still don't really know exactly why but we did find
out that is the compatibility issue with multiple an event that an event that was the underlying and acing framework that I'll of uh with framework was using and it was causing some craziness another problem we we ran into was when using AI over HTTP and was that you know once a week this kind of and so that we had running uh just kind of ran out of memory and and crashed and the for do it some specific kind of indentation details that we didn't have no complete here if you understand and we had to kind of release the object uh the response object if there was an error otherwise memory to some of things around them and this is the stuff that you really don't have be dealing with but we need a lot of the things that the Python you probably gonna find of dealing with this kind of like weird things around again so the last thing I would say that the mean to be controversial at like Europe might contain don't look to Python for your life remains sense of engineering achievement that so my of Python is is very much like a unstable the incremental relationship where like a move from being infatuated to being totally in love on like a daily basis using really cool stuff like generators and decorators and context managers and proteins in all this stuff and but then I would get something like this so this is just a function that returns think and it's and if I was reading this for the 1st time what I would imagine is that the you'll think statements technology and it was returned to the and everyone's happy and it all makes sense but doesn't do that so it becomes generator function returns generator so again whatever I can kind of reset button if you go make some this generator I guess they were there for the 1st time on that maybe the attendance gifts and it yields to you but no fact racist stuff iteration and with the value of the return and if you got a good reason that is a paper for this this but do with proteins and some generators and stuff like that but to a new user it just this just says returned doesn't mean 1 thing in Python and and this is kind of a this it's stuff like this that I have to keep reminding myself that Python service characteristics of personality this is a quote from within a Python ends essentially it says practicality periods and this is really important to us because we all way way more concerned about making health care affordable and accessible then we are about doing it in the most pure way and that Python has the very same approach Ridgeway more interested in solving the problem and in getting things done and that is about doing it in the most pure spatial way which is very different to when you go to the scholar Ghazal elixir guys and stuff like that and with that approach we've been able to build something that's really really amazing at picking people's lives on a daily basis and will make way more interested in that then we are about putting it in the most pure way and python allows us to kind of do that and which is something that is really amazing to us so that's it for me and what's next for us we were moving into diagnostic space and room to Weimar comprehensive of prediction and monitoring and space remote much of what international which I can tell you is not trivial so like natural language reneges on natural language for Arabic is like 4 and
Chinese like and the so that's on
any questions thank
you with just 1 short question please the really interesting talk um did you for is just thought take into account social situations medical history of the patients asking a certain question because you can have a lot of on the questions of the will ask after he knows about the history of the patient what about the social context so I didn't quite catch the question creation for example when a patient asks have added what what should I do again really matter if the patient is a young boy who goes out and so that we had so at the moment something so we
we read that like maybe 2 months ago no but recent years and so recently we we we take uh demographics and uh age and gender on the of kind of you know Ottoman offered you gonna take that you're going to and and what Brooklyn knowledge of not just take that but also thing everything known about you know armies vegetated they did the medication thick wall into the kind of advice giving to you and we we have now definitely that adding these more kind of complex reasoning deductions to and so restored about that of Figure mushroom during we have