6th HLF – Laureate Lectures: Autonomous Systems – A Rigorous Architectural Characterization


Formal Metadata

6th HLF – Laureate Lectures: Autonomous Systems – A Rigorous Architectural Characterization
Title of Series
Sifakis, Joseph
No Open Access License:
German copyright law applies. This film may be used for your own use but it may not be distributed via the internet or passed on to external parties.
Heidelberg Laureate Forum Foundation
Release Date

Content Metadata

Subject Area
Joseph Sifakis: "Autonomous Systems – A Rigorous Architectural Characterization" The concept of autonomy is key to the IoT vision promising increasing integration of smart services and systems minimizing human intervention. This vision challenges our capability to build complex open trustworthy autonomous systems. We lack a rigorous common semantic framework for autonomous systems. There is currently a lot of confusion regarding the main characteristics of autonomous systems. In the literature, we find a profusion of poorly understood “self”-prefixed terms related to autonomy such as Self-healing, Self-optimization, Self-protection, Self-awareness, Self-organization etc. It is remarkable that the debate about autonomous vehicles focuses almost exclusively on AI and learning techniques while it ignores many other equally important autonomous system design issues. Autonomous systems involve agents and objects coordinated in some common environment so that their collective behavior meets a set of global goals. We propose a general computational model combining a system architecture model and an agent model. The architecture model allows expression of dynamic reconfigurable multi-mode coordination between components. The agent model consists of five interacting modules implementing each one a characteristic feature: perception, reflection, goal management, planning and self-adaptation. It determines a concept of autonomic complexity accounting for the specific difficulty to build autonomous systems. We emphasize that the main characteristic of autonomous systems is their ability to handle knowledge and adaptively respond to environment changes. A main conclusion is that autonomy should be associated with functionality and not with specific techniques. We conclude that autonomy is a kind of broad intelligence. Building trustworthy and optimal autonomous systems goes far beyond the AI challenge. The opinions expressed in this video do not necessarily reflect the views of the Heidelberg Laureate Forum Foundation or any other person or associated institution involved in the making and distribution of the video.

Related Material

Turing test Internet forum Autonomous System (Internet) Musical ensemble Model checking Formal verification System programming
Machine vision Autonomous System (Internet) Thermodynamic system Service (economics) INTEGRAL Machine vision Disintegration Predictability Internet forum Mathematics Smart card Computer network Object (grammar) System programming Directed set Identity management
Dynamical system Process (computing) Thermodynamic system Service (economics) INTEGRAL Machine vision Interactive television Model theory Axiom Parameter (computer programming) Group action Mereology Sequence Revision control Internetworking Order (biology) Internet der Dinge Units of measurement Resultant
Standard deviation Machine vision Building Model theory Basis (linear algebra) Computer font Thermodynamic system Systementwurf Response time (technology) Video game Mathematics Machine learning Dependent and independent variables Information security Control system Service (economics) Product (category theory) Thermodynamic system Software developer Fitness function Electronic mailing list Perturbation theory Term (mathematics) Functional (mathematics) Maxima and minima Systems integrator Internet service provider System programming Information security Electric current Pressure Autonomous System (Internet) Product (category theory) Service (economics) Disintegration Mobile Web Virtual machine Rule of inference Telecommunication Internetworking Term (mathematics) Software Microprocessor Computer hardware Integrated development environment Plug-in (computing) Systems engineering Standard deviation Model theory Division (mathematics) Evolute Limit (category theory) Component-based software engineering Software Function (mathematics) Computer hardware Contrast (vision) Autonomic computing Computer network Kolmogorov complexity Communications protocol Force Systems engineering
Autonomous System (Internet) Group action Algorithm State of matter Graph (mathematics) Robot Infinity Thermodynamic system Tablet computer Object (grammar) Network topology Green's function Reduction of order Control theory Physical system Robot Game controller State transition system Thermodynamic system Electric generator Modul <Software> Model theory Planning Automaton Set (mathematics) Group action Computer Logic synthesis General relativity Network topology System programming Finite-state machine Object (grammar) Game theory Computational visualistics
Computer chess Axiom of choice State of matter Model theory Decision theory Parameter (computer programming) Mereology Thermodynamic system Medical imaging Video game Repository (publishing) Automation Control theory Data storage device Information Process (computing) Bounded variation Descriptive statistics System dynamics Identity management Enterprise architecture Thermodynamic system Process (computing) Constraint (mathematics) Reflection (mathematics) Optimization problem Bit Instance (computer science) Functional (mathematics) Virtual machine Category of being Numeral (linguistics) Fluid statics Computer configuration Data storage device Repository (publishing) Control theory Configuration space output Personal area network Condition number Data type Boiling point Autonomous System (Internet) Read-only memory Line (geometry) Run time (program lifecycle phase) Robot Channel capacity Adaptive behavior Virtual machine Planning Annihilator (ring theory) Device driver Goodness of fit Whiteboard Autonomic computing Energy level Configuration space Maize output Subtraction Game theory Robot Game controller System dynamics Computer chess Information Machine vision Model theory State of matter Mathematical analysis Planning Independence (probability theory) Set (mathematics) Device driver Partial derivative Computational visualistics Identity management
Autonomous System (Internet) Enterprise architecture Thermodynamic system Model theory Model theory Virtual machine Reflection (mathematics) Parameter (computer programming) Mereology Functional (mathematics) Thermodynamic system Virtual machine Degree (graph theory) Factor analysis Systementwurf Degree (graph theory) Component-based software engineering Autonomic computing Energy level Energy level Kolmogorov complexity Object (grammar) Data type Identity management
Dynamical system Graph (mathematics) Texture mapping Model theory Decision theory Finitary relation Semantics (computer science) Thermodynamic system Formal language Mathematics Component-based software engineering Type theory Synchronization Object (grammar) Repository (publishing) Vertex (graph theory) Information Multiplication Position operator Enterprise architecture Collaborationism State transition system Thermodynamic system Electric generator Process (computing) Mapping Organic computing Reflection (mathematics) Electronic mailing list Automaton Instance (computer science) Functional (mathematics) Arithmetic mean Repository (publishing) Telecommunication System programming Configuration space Pattern language Data type Autonomous System (Internet) Trail Enterprise architecture Motif (narrative) Reflection (mathematics) Distance Rule of inference Declarative programming Goodness of fit Computer hardware Data structure Address space Form (programming) Motif (narrative) Mobile Web Rule of inference System dynamics Information Model theory Physical law Interactive television Set (mathematics) Semantics (computer science) Component-based software engineering Word Function (mathematics) Atomic number Synchronization Vertex (graph theory) Object (grammar) Computational visualistics Invariant (mathematics) Address space
Autonomous System (Internet) State observer Spacetime Implementation Divisor Model theory Decision theory Multiplizität <Mathematik> Event horizon Thermodynamic system Factor analysis Medical imaging Systementwurf Latent heat Autonomic computing Control theory Process (computing) Information output Information security Mathematical optimization System dynamics Thermodynamic system Information Decision theory Planning Volume (thermodynamics) Instance (computer science) Prediction Functional (mathematics) Event horizon Hausdorff dimension Autonomic computing Partial derivative Right angle Kolmogorov complexity Information security Data type Mathematical optimization Identity management
Systementwurf Thermodynamic system Component-based software engineering Divisor Model theory Autonomic computing Interactive television Coordinate system Complex system Kolmogorov complexity Object (grammar)
Spacetime Code Length Quality of service Thermodynamic system Independence (probability theory) Component-based software engineering Object (grammar) Dependent and independent variables Repository (publishing) Code Information Physical system Cybersex Enterprise architecture Thermodynamic system Physicalism Streaming media Functional (mathematics) Fluid statics Resultant Spacetime Autonomous System (Internet) Domain name Computational visualistics Motif (narrative) Transformation (genetics) Parametrische Erregung Cybersex Streaming media Coprocessor Number Architecture Term (mathematics) Configuration space Game controller Rule of inference System dynamics Information Tape drive Interactive television Line (geometry) Coprocessor Group action Single-precision floating-point format Embedded system Personal Assistant Kolmogorov complexity Internet der Dinge Object (grammar) Identity management
Autonomous System (Internet) Enterprise architecture Service (economics) Parametrische Erregung Ring (mathematics) Programmable read-only memory Mobile Web Verteiltes System Thermodynamic system Number Usability Architecture Web service Mathematics Component-based software engineering Cache (computing) Read-only memory Computer hardware Subtraction Sanitary sewer Mobile Web Robot Enterprise architecture System dynamics Thermodynamic system Spacetime Organic computing File format Forcing (mathematics) Client (computing) Ring (mathematics) Instance (computer science) Computational complexity theory Component-based software engineering Web service Telecommunication System programming Kolmogorov complexity Data type Asynchronous Transfer Mode
Constraint (mathematics) Thermodynamic system Web 2.0 Systementwurf Video game Type theory Read-only memory Data storage device Renewal theory Computing platform Mathematical optimization Self-organization Standard deviation Thermodynamic system Sicherheitskritisches System Software developer Food energy Instance (computer science) Group action Coprocessor Subset Word Error message Order (biology) Authorization Data type Mathematical optimization
Autonomous System (Internet) Model theory Mountain pass Time zone Virtual machine Volume (thermodynamics) Thermodynamic system P (complexity) Systementwurf Term (mathematics) Formal verification Authorization Process (computing) Category of being Mathematical optimization Proof theory Machine learning Zoom lens Standard deviation Thermodynamic system Process (computing) Trail Model theory Limit (category theory) Logic synthesis Number System programming Formal verification Software testing Kolmogorov complexity Object (grammar) Mathematical optimization Address space
Point (geometry) Autonomous System (Internet) State of matter Dot product Data recovery Auto mechanic Mathematical analysis Event horizon Thermodynamic system Predictability Number Systementwurf Tablet computer Causality System identification Subtraction Covering space Thermodynamic system Graph (mathematics) Hazard (2005 film) Data recovery Direction (geometry) Model theory State of matter Open source Mathematical analysis Mechanism design Database normalization Embedded system Event horizon Network topology Crash (computing) System programming Kolmogorov complexity Data type Electric current
Standard deviation Building Context awareness Transportation theory (mathematics) Model theory Decision theory File format Public key certificate Thermodynamic system Predictability Independence (probability theory) Systementwurf Machine learning Personal digital assistant Software framework Machine learning Product (category theory) Decision theory Building Software developer Term (mathematics) Functional (mathematics) Fluid statics System programming Software framework Software testing Mathematical optimization Electric current Ocean current Point (geometry) Autonomous System (Internet) Product (category theory) Asynchronous Transfer Mode Robot Rule of inference Goodness of fit Latent heat Software Software testing Implementation Subtraction Game theory Game controller Robot Rule of inference Standard deviation Event horizon Function (mathematics) Autonomic computing Kolmogorov complexity Game theory Mathematical optimization Systems engineering
State observer Musical ensemble Decision theory Outlier Real number Virtual machine 1 (number) Control flow Revision control Stochastic Casting (performing arts) Video game Formal verification Elasticity (physics) Control theory Software testing Extension (kinesiology) Subtraction Condition number Form (programming) Addition Standard deviation Thermodynamic system Process (computing) Information Model theory Basis (linear algebra) Perturbation theory Line (geometry) Instance (computer science) Entire function Internet forum Arithmetic mean Mathematical optimization Resolvent formalism Row (database)
it's my pleasure to introduce the second speaker of the morning sessions or George C pockets he got the Turing award in 2007 for his role in developing model checking as highly efficient verification tool and in his talk today he will talk about I think a very hot topic naming about autonomous systems and what are the challenges and perhaps some of the overseen challenges in designing them so the title of his talk is autonomous system rigorous characterization of the architecture and
technology is working sir thank you q last year I talked about system design so it will continue the tradition and this year I will focus on autonomous systems the main motivation for parties a vision that we call the Internet of
Things and this a vision that promises marvelous a lot of marvelous achievements integration of services math services smart management of resources and last year he had explained that this vision involves in fact two
parts of an even difficulty something especially is called the human Internet of Things where it's a mere improvement of the internet you have the basic model is ok you have you submit a request and you get some answers and this new version of the Internet there will be semantically more each and the most challenging part is the industrial Internet of Things where here we are seeking full integration of services and systems without human intervention so we want to fully automate the processes and humans will only tune some parameters and this is really the challenging part of the Internet of Things there is
currently a an important industrial consortium that works towards developing infrastructure and solving all the problems at raises this vision frankly speaking I have not seen any interesting result so far the problems we are facing in order to achieve this are very very
hard because autonomous systems are critical so well understood difficulties that we have infrastructures that are not trans worth enough in particular they are not secure enough so security is a very big issue and we don't know how to cope with that also another problem is the impossibility to guarantee response times if you want to have control loops you should have guarantees that guarantee times and and protocols are synchronous you cannot do that in the with the existing network infrastructure and there are other problems that come from the fact that you tighten degrade have mixed with ecology system integration very critical systems with systems that are may be vulnerable and I hope you can imagine this now these are the limitations and I would like also to emphasize that despite these limitations and the repression of markets we have new practices you know that today functionality can change in systems so hardware is a kind of commodity and then you can create the new services my add-in software software can be mobile a configurable customizable and this is a standard practice now in the internet and this is practice can become very dangerous going to apply to critical systems like autonomous systems you probably know that Tesla can software may be updated every month okay so this is a very dangerous practice I have worked on political systems for decades now and there are rules that require that we don't change once a product is put in the market and this is here some criticality it is certified and the product cannot be changed so what we are doing for Tesla cars and other devices when in a chance the software is against standard practice let me remind you that when an aircraft is certified you cannot change anything not only the software but you buy all the hardware that is needed for for a life of the aircraft so you have these new practices and this is a reality and quite soon we will have self-driving cars and I believe that this is an inevitable evolution and the problem is how a systems engineering will change to to meet these needs we are moving from small sized centralized systems that are known evolve evolve up and are automated to autonomous systems that are distributed so in systems I have designed for aircraft the environment is very predictable and now for say self-driving cars we have dynamically changing environments and also when we design critical systems we have the ambition to guarantee correctness to provide guarantees at design time and I think that we should break whoo-ha it's not anymore possible to provide all guarantees of correctness design so these are you division challenges our capability to build trans coffee systems and I think that for autonomy we need a technical definition of autonomy based on some semantic model not just list of terms self prefix term so many papers in the literature about autonomy system you see that an autonomous system is self-healing self of the myself protect itself everywhere whatever okay and this technical definition should allow us also to understand how we can enhance systems autonomy and for its enhancement what are the risk we take what are the technical difficulties and I would like also to remark that the technical discussion about the autonomous vehicles focuses mainly on AI and machine learning I don't doubt that these technologies are important or key but to achieving that autonomy nonetheless there are other issues related the design of the systems and if our design has flaws no matter how intelligent is this the machine learning techniques you use okay it does not help so we should be careful with that very important question I'd like to address here is whether it is possible to develop rigorous design methodologies and associated Transcorp fitness assessment techniques so this is an outline of my
talk I will introduce the concept of autonomy then I would like to show you how it is possible to have a model-based approach why model-based approaches are important because you need the models of the system you design to establish guarantees if we don't have model then you cannot have guarantees and then after that I will try to explain how the Guru's system design approaches that are located by standards can be applied to autonomous systems or hot lava difficulties to do that so to explain
the concept of autonomy we'll consider five exams thermostat trained shot a chess-playing robot a soccer playing robot and the robocar so all these are systems men consist of ages just introduce some terminology an agent is a computing system and the agents control objects these are physical systems and they interact in an environment and the purpose is to meet some global goals now as a designer if I have to design an autonomous system I have to determine that the behavior of the agents pursuing individual goals so that the collective behavior meets the system global goals I think that the problem at least is clearly stated now I would like to open a parenthesis here and explain what it means technically to meet goals so this again plays the agent for a given environment no the agent knows the goal so for me the environment model is an
automaton is a transition system so you have States you have actions and in this model you distinguish between green actions during actions are controllable actions that can be controlled vary by the agent and the reductions that are uncontrollable can be played by the environment so if I have a consider this example I have goals never it's bad so bad is a state of my model eventually reach target this is another state so I can where are methods that synthesize plans the plan is a tree in the most general relation a tree with alternating controller and controller action so generating plants is like solving game problem and okay I can say a lot about that this problem of plan generation does not admit algorithmic solutions unless the environment model is simple enough it is say finite state in that case you can synthesize a maximal controller that contains all the possible plans but in general this is a non computable problem so now if I go back to the set of exams
I have considered okay you see the differences regarding the environment very simple to understand the stimuli so for the thermostat the similar very simply just numerical information temperature for vision sharply it's a bit more more complicated it's dynamic configuration of cars and state of equipment for the chest robot stud configuration of bones you can read and then main difference are the way we deal with goals so for a thermostat it's trivial to have an explicit controller a single goal for the Sadler I have design controls for for shuttles of this type you have an explicit controller so I design time you can compute a controller and then you have some online adaptation techniques for or comfort for instance many fixed goals and then as you move downwards you see that you cannot have static plan you have online planning and for chess robot you store knowledge for sake robot also some stored and generated knowledge and have dynamically changing goals so I hope that through this set of exams you see that I can have automated systems and the difference between automation and and autonomy will become more clear when I
show you what I consider to be a good agent model so an agent interacts with an environment through our sensors and actuators or the sensors sent the sensory information this is the input to the agent and the agent sends commands that are sent to our actuators may change the state of the environment so let here this architecture involves five distinct functions and independent functions and one knowledge a repository knowledge is very important for agents and for autonomy so the knowledge repository in the knowledge repository store concepts what I call concepts are models of of the objects and of the agents of the environment and their properties and this is very important for instance for this function the perceptual function the perception function takes sensory information and identifies analyzes and says I seek a car a pedestrian a traffic light so concepts that are stored in the knowledge repository then the other function is the reflection function where all of the reflection function is to build a model of the environment so I build a model of the environment why I need the model of the environment because I want to make decisions based on this model and they make decisions by combining two functions goal management and planning what means goal management I have a set of goals and the goal management means boils down into a solving an optimization problem where I under some constraints that are constraints on available resources for instance have the best quality of computation under deadline constraints or under our constraints of memory or or of image and then the planner given a goal given a model of the environment generates plans and then what is very important for adaptation is this self adaptation function that handles knowledge where we find knowledge knowledge is generated either by using machine learning techniques or analysis techniques on the model the environment model and so the self adaptation self adaptation means that I am able to chance the goal management process either by changing some parameters that count in the choice of goals in the manual management process or by changing goals so this is a first description of the object-ball model life I hope this is it's clear how you can combine these five basic functions and here I have a definition of autonomy that that comes from from what I said so far so if we consider this model we can now understand the difference between autonomous and automated it's clear that the thermostat is not an autonomous system why it's not an autonomous system because you don't need any one of these functions okay you have static control a design time you don't have goal management you don't have any planning faction self adaptation etc so another important question is that in some systems you have some part of the autonomy is ensured by automated systems and dollar by humans and this is a concept that is important in autonomous systems engineering is called the autonomy level of the system so here I am giving you an example of autonomy levels for self for cars and this is the the definition of autonomy levels by the Society of Automotive Engineers so you see that you go from level zero to zero five zero five is for full automation so no drivin wheel no steering wheel at all live level four is high automation so for instance self-driving cars with a specific equipment in say what we call automated highways level three is an interesting level because here the car has direct direct the system has the control and upon systems request the human driver must be prepared to respond this is this is a situation how to manage so you have this concept of system autonomy levels and if I use the
the architecture I have proposed the concept I have the concepts I have proposed I can characterize autonomy levels so the part that is automated by machine and the part of these functions that is assisted by humans ok so I hope that what I said is clear and in practice I would like to emphasize that to determine the autonomy level of a system we have three types of parameters one is what I call autonomic complexities I will define this concept later the other is design complexity and the third is the degree of Trance worthiness that that you are seeking so typically if you have critical autonomous systems it's not realistic today to have full autonomy you will find some compromises where humans can intervene ok so now let me consider
something more technical how I can define models for systems and what are the minimal concepts to define models for autonomous systems now there is a problem with okay I don't know why some instability of the system okay so let me explain this idea that how I can define model sky can define behavioral models of autonomous systems why I need models as a designer I should understand how objects and components and agents interact and I ensured for each agent I should
insurgents should be able to define a model of its environment so I should explain what is a good model for for for autonomous systems I've been working for a few years on this idea now we have a language to describe autonomous systems and given some the basic concepts of this language in fact we used to describe autonomous system so a system I consider that it is composed it's a set of motives motives its ownership and architectural motif it's a hardware component instances live so component is that I suppose that I have types of agents and objects so these are instances of agents and objects and I suppose that each motif is equipped with a map a map is a reference structure that is the place a very very important role for the organization of the computation and the components of the motif that may belong to many different motives have addresses that are nodes of this map through matters function so this act function defines the positions of all the components in the map and then we can talk about interaction between components and interaction means strong synchronization between components collaboration between and configuration rules configuration has to do with the beginning of dynamism of the system so you can have mobile components what it means by the components the addresses of components may charge creation deletion of components or the even dynamic change of the map okay so we have defined the language and the meaning of these models can be defined by using operational semantics is an automaton is a transition system from on which I can
apply tracking generation of plants now this is an example I'm showing you we have say self-driving cars on a road and I have here two maps two motifs one the road trunk map which is a geographic map of which I represent I show the addresses the nodes are coordinates geographic coordinates of the road I'm a communication map and then I can write in my language interaction rules of this form for instance that for any vehicles a and a prime if their distance is small enough then they exchange their speed or I can have mobility rules that are configuration rules so I have a language to do that and the SIS I have a language to do that and now I know how to define the behavior of agents so the behavioral variants of course when I design the system I should have a clear understanding about what is the environmental model and so each engine receives as I said sensor information and perceptor identifies agents and object types and give this information to the reflection function that builds the model and of course the problem is how to build dynamically a model and keep track of dynamically of the challenges of the environment and this is this is a very very important problem I would like also to say a few words about the use of knowledge because this is a very important aspect first of all you should have a formal definition of what knowledge is and knowledge is about the definition of types of agents and objects map patterns and also some declarative knowledge laws invariance of the system methods you use so all this you have so list of items here that characterize the knowledge of the repository and of course how efficient is this interaction between either the agent and the environment depends on how you use knowledge to make in the decision process okay now I would like
to discuss this issue that is very important how much hard is to build autonomous systems and I would like to show that this takes prospects one is autonomic complex is so issues that are strictly related to autonomy so to the realization of a function same I mentioned and there is another dimension that is completely independent that is design complex so there is a third type
of complexity I do not talk about is implementation complexity this is quite standard so let me say what is autonomic complexity autonomic complexity can be characterized by these factors here complexity of perception how difficult it is to interpret the stimuli to cope with ambiguity for instance of the images you get or the vagueness of the images you get also how what is the volume of data you have to analyze to extract some relevant information also another factor is the lack of observability and controllability how much you control and you observe in your environment the more you increase of the observability and and controllability the better to make the right decisions another factor is uncertainty uncertainties means the lack of predict ability lack of predictability and this may have various reasons and of course most critical events are a predictable failures at our consetta and finally complexity of goals there are goals that are easy to manage for instance safety goals reachability goals are much harder or security or the plan optimization of resources goals are much harder to to to to to achieve and okay so this is gives you an idea about what problems what of a difficulty specific to or autonomous behavior now
this should be emphasized and understood when you design a system an autonomous
system this is a complex system has components components are agents and objects and these should be coordinated so there are two factors in design complex one is reactive complexity of components I'm going to explain this and the other is architectural complexity how much it costs to coordinate components and reactive complexities how much involved is the interaction of components with with very very so
reactive complexity does not characterize the complexity in space and time or whatever the length the number of lines of code in the component characterizes the intricacy of its interaction with the environment so you may have very simple components in the sense that their interactions very simply understand the environment transformational components you may have streaming components streaming component you have streams of values and you produce streams of value so here you care about functionality and also about the latency about latency typical exams are say encoders signal processor signal processing systems and then you have embedded components and that these components are components that continues interact with an external environment so here the goals you have to maintain our much more complicated functional goals but we have a quite what we call quality of service goals and finally and this is very important for autonomous systems in the internet of things you may have cyber physical components so I said the physical component integrates I mean it's an embedded component with its own environment in terms of objects so it's abra physical component is receives not only inform information about about i mean--can receives also information about the physical quantities and and that's very important so physical said the physical components you know if you want now to represent them as agents you should consider not only the external environment but the internal environment and I should say that cyber physical systems is a domain of results that is a
very active current now how about architectural complexity ok so you can figure out a way to go from simpler architectures to more involved architectures by considering this this classification that comes here from my architectural model and here is what I
propose as a classification you may have a static architectures so as a hardware architectures where you have components and all the interconnect is known and more difficult more involved these are parametric architectures like in a ring architecture in a ring architecture you may want an architecture that works for any kind and number of components in the ring and then dynamic architecture is a parametric architecture where you can create and delete agents mobile architecture like for instance architectures needing while telecommunications you have these are dynamic architectures where you have a mobility so so they're the others function changes and finally the most complicated architectures are self-organizing architecture we have many motives many modes of organization like for instance ease of driving cars so just to show you what you get if you
consider different types of systems in this space architecture complex theory are reactive complexity so you see that complexity increases as you go up and right it's interesting also to note this distinction between services and systems so services you use for service systems form a useless formation and streaming components and force for systems embedded and cider components so the place where you have increased autonomy is there so this these are all aspects also to be considered when you design autonomous systems ok so let me finish
by saying a few words about a renewal system design what this is a concept that rigorous system design is required by standards for political systems and you know that when you build systems one concern instance covenants that the system would behave correctly despite any problems that can encounter in its life and optimization goals of course because if you give me a lot of money I can give you a very fast quality system you cannot sell it so ok and you understand that the two types of goals are antagonistic so today we have a
situation where we have political systems this is the development of this systems cost a lot best-effort systems so these are very large systems and some intermediate situations now what you'd like to emphasize is this that when you build a flight controller very liabilities 10 to the minus nine failures per hour and the single failure is not catastrophic and the reliability of some platform is the web platforms say ten to the minus four why we don't have everywhere reliability ten to the minus nine okay the answer is simple even for rockets if you want to multiply by ten the reliability of a rocket you will have to pay a thousand times money more okay so when you tie and this is a problem we are facing today in order to
have large autonomous systems that may be safety critical when we apply rigorous design techniques they are based mostly on verification verification means that you try to get guarantees you have a model of the nominal behavior of the system the goals you try to formalize the goals and you reason about that and you try to provide guarantees I don't want to give details because most most people are familiar with the verification but you have somehow to formalize the goals and for for autonomous systems complex
autonomous systems I let you contemplate this is these are goals for self-driving cars we have a 28 you cannot read but just to show you some of these goals these are very hard to interpret in terms of very concrete objectives and to formalize for reasons I hope you hopefully understand so verification as we have applied it today in standards suffers some limitations that I list here and ok probably I don't have time how much time I have left 10 minutes yeah seven another seven okay so you have well-known limitations and also something else that should be remarked is that even if you prove your system correct truth is a social process okay even if I say I have a system it is perfect I have proven it with my program okay this should be confirmed by an authority and that's a problem furthermore I would like to emphasize that machine learning techniques cannot be verified cannot be formally verified why it cannot be firmly verified because you know to verify you need the requirements and we don't have the requirements about how I cut is different from a door so okay so let me
skip this and once now you provide guarantees then you have so when you provide guarantees you provide guarantees for a nominal behavior of our nominal model and then you have to take into account all the assumptions you made about the some ideal behavior of the environment so the nominal model says what are the transwarp Enda non-trans wealthy states and then you should consider hazards of different types you do what we call risk analysis and all of this risk analysis is to determine amount now the non-trans graph estates non-fertile states and fatal states the idea is that by using redundancy techniques when a harmful event occurs you will not go into a fad of state a further state is a state where the trans worthless is fully compromised okay you cannot go back and there are techniques to do that and we know how to apply this techniques for small systems I have done this for our UNIX systems so you have a recovery mechanism now these techniques just to summarize are not applicable to complex autonomous systems and you can imagine the reasons one reason is that these systems are to complex we cannot pray we cannot analyze just consider an example here
this is this is an example of failure topology cover in ninety nine point four percent of the light the clashes for some some number of cases and what means the vehicle failure the failure should be analyzed because has many different causes and according depending on the course we can have recovery techniques so these systems are extremely
complicated to apply the standard techniques and I would like to close with a discussion two aspects one I
think that the important question is whether we will have standards we will have guarantees current standards cannot be applied so today self-driving cars and the other products are self certified which sounds like a joke so the manufacturer says my car is good enough just use it ok no independent certification so the important question is whether it's possible to guarantee safety just by testing so companies like Tesla or where you more say just testing if it's tested enough I don't think that this this this is technically acceptable there is an interesting paper by mobile I which is a subsidiary of Intel that advocates model-based techniques that's a good point and of course the chaotic the question is whether it is possible to develop rigorous design techniques this is an open question now in what I said I think I clarified the concept of autonomy I think that autonomy should be associated with functionality and not with specific techniques as many papers do that so machine learning is essential but it's not the only way to build receptors and adaptive controllers and I consider that autonomy is a kind of broad intelligence it's not just decision so there is a big difference between an autonomous vehicle at a game playing a robot so if people tell me that alphago is the Summum is the height of intelligence I love okay it's ridiculous to say this because we know how it works and I think that it's not true because in that case you have a very good situation awareness you know perfectly the rules of the game and in autonomous systems you don't know the rules of the game cannot formalize them so building transporte and optimal autonomous systems goes far beyond the current ia challenge thank you [Applause] thank you very much for this very interesting talk showing us the challenges we face in our future so are there we have time for a few questions aren't they yes thank you for the amazing talk so I would like to know what a nice but is there any standard frameworks in which we can introduce Tok
asta city in the entire verification process because most of the real life environments for example the ones in which autonomous cars learn and act are highly stochastic so how do you make a model like model the stochasticity in its full-fledged form thank you okay yes okay so in the current standards regarding guarantees are considered some that very deterministic approach now I think that we should introduce the cast elastic approaches but this still should be model-based okay not just saying I have tested for I don't know how many millions of hours my system and it behaves correctly without being model-based okay so when you test for instance a program we are still model-based you have a criterion of coverage for instance so models are very important we cannot ignore models and the problem with all all these machine learning techniques AI techniques is that they refuse I mean they don't know how to introduce models and models in many minutes of the basis in any scientific approach all scientific knowledge is based on the use of models okay so we should not I mean we should understand this now this can be perfectly stochastic whatever but based on models so it seems to me the the biggest problem with self-driving car technology is what Alko outliers which is they depend on the white lane the white lines on the road being there when the city comes along and repave z-- the road then the white lines are gone for a couple days in huge snowstorms the the sensors fail and it seems to me you know if this solar glare then their sensors fail and it seems to me that to make things reliable you need to somehow figure out what all the outlying conditions are and figure out how to test against them and that seems completely impossible yes what do you do is yes you tell you what is for me a realistic approach that can be applied today provided we make the investments for that realistic approaches that we equip roads with the adequate infrastructure that is not affected by perturbations you mentioned okay and this is realistic to have in highways for instance to equip a highway I don't know San Francisco Los Angeles whatever or there are some interesting experiments in Europe this is the realistic approach I don't believe that it's realistic to have a mix of self-driving and human driven cars in cities okay this may be a catastrophic so I I think that we should be careful about that but there are technical solutions if if we have enough observability and controllability and this is reliable then I think that we can solve the problem it it seems to me that at least conceptually there's a difference between trying to design autonomous vehicles that say look at all the other vehicles and look at the pedestrians and look at the environment and make decisions about what to do compared to a system where there might be a bigger agent that's looking at all the vehicles or at least all the vehicles in some region and is trying to optimize things like traffic control in addition to safety and I'm wondering is this really a fundamental difference or is there is one just a simple extension let me summarize if I understood your question one is the situation where you have autonomous vehicles each one making its decisions and pursuing its own goals and the other is to have some monitoring system global knows the global picture and resolve conflicts it's much simpler but the problem is that you have geographically distributed vehicles I think the solution will be to have some monitoring system to make things more predictable it's a matter also of real-time ok how much real-time can be a centralized system that centralizes the information analyzes it because decisions in some cases must be taken really on the spot I remember with very short reaction times but I think that the compromise is to have a solution so it kept centralized solution on fully distributed solution and decentralized solution so the centralized means that you have some arbitration processes that are supervising this this is a realistic solution but we see we'll see there is a difference yes questions for the lunch breaks or come back after the talk in front and the rear spoiler has an Altman to make but before his announcement let's thank rows of C pocket [Applause] [Music] [Music] you


  781 ms - page object


AV-Portal 3.9.2 (c7d7a940c57b22d0bc6d7f70d6f13fde2ef2d4b8)