Transhuman Expression

Video thumbnail (Frame 0) Video thumbnail (Frame 3419) Video thumbnail (Frame 4673) Video thumbnail (Frame 5961) Video thumbnail (Frame 7169) Video thumbnail (Frame 8475) Video thumbnail (Frame 9773) Video thumbnail (Frame 10996) Video thumbnail (Frame 12277) Video thumbnail (Frame 13662) Video thumbnail (Frame 15524) Video thumbnail (Frame 16879) Video thumbnail (Frame 18074) Video thumbnail (Frame 19299) Video thumbnail (Frame 20628) Video thumbnail (Frame 22013) Video thumbnail (Frame 24038) Video thumbnail (Frame 25350) Video thumbnail (Frame 28183) Video thumbnail (Frame 30226) Video thumbnail (Frame 32769) Video thumbnail (Frame 34203) Video thumbnail (Frame 38238) Video thumbnail (Frame 40100) Video thumbnail (Frame 41444) Video thumbnail (Frame 43163) Video thumbnail (Frame 44539) Video thumbnail (Frame 45811) Video thumbnail (Frame 47441) Video thumbnail (Frame 49924) Video thumbnail (Frame 51131) Video thumbnail (Frame 52581) Video thumbnail (Frame 54210) Video thumbnail (Frame 55983) Video thumbnail (Frame 57317) Video thumbnail (Frame 58575) Video thumbnail (Frame 59785) Video thumbnail (Frame 62340) Video thumbnail (Frame 63569) Video thumbnail (Frame 66737) Video thumbnail (Frame 68012) Video thumbnail (Frame 72257) Video thumbnail (Frame 74852) Video thumbnail (Frame 76545) Video thumbnail (Frame 77964) Video thumbnail (Frame 79875) Video thumbnail (Frame 84392) Video thumbnail (Frame 87484) Video thumbnail (Frame 89177) Video thumbnail (Frame 90466) Video thumbnail (Frame 91732)
Video in TIB AV-Portal: Transhuman Expression

Formal Metadata

Title
Transhuman Expression
Subtitle
Interdisciplinary research in Painting and Robotics. with Prof. Oliver Deussen, PhD candidate Marvin Guelzow, and the Artist Liat Grayver.
Title of Series
Author
License
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2018
Language
English

Content Metadata

Subject Area
Abstract
The meeting point of art and science as a place of inspiration, exchange of knowledge and creation is the main focal point of the talk. Together with Prof. Oliver Deussen, the PhD candidate Marvin Guelzow, and Liat Grayver we will discuss both the technical challenges and innovation aspects in the development of the e-David robot, alongside the the social and artistic practice its offers. Topics as such “paradigms of creativity” under the title “New Materialism / Anthropocentrism / Posthumanism” will be presented with the goal to position and understand machine-assisted creative interfaces within the broader field of media art and painting traditions.
Keywords Art, Culture

Related Material

The following resource is accompanying material for the video
Video is cited by the following resource
Suite (music) Robotics Personal digital assistant Software developer Factory (trading post) Expression Square number Control flow Musical ensemble
Diffuser (automotive) Musical ensemble
Robot Medical imaging Robotics Multiplication sign Sound effect File viewer Mereology Bounded variation Formal language
Quantum state Orientation (vector space) Multiplication sign Chaos (cosmogony) Bit Neuroinformatik Uniform resource locator Logic Order (biology) Logic Entropie <Informationstheorie> Energy level Bounded variation Physical system
Point (geometry) Digital photography Medical imaging Computer-generated imagery Computer-generated imagery Artistic rendering Virtual machine Artistic rendering Computer Bit Mereology Computer graphics (computer science)
Decision theory Compiler Multimedia Mereology Neuroinformatik Physical system
Robot Robotics Moving average Musical ensemble Neuroinformatik Physical system
Observational study Multiplication sign GUI widget Virtual machine Mathematical analysis Sphere Virtual machine Digital photography Process (computing) Robotics Different (Kate Ryan album) Hash function Normed vector space Process (computing) Musical ensemble
Area Robotics Forcing (mathematics) Feedback Virtual machine Multimedia Parallel port Musical ensemble
Process (computing) Combinational logic Computer Process (computing) Musical ensemble Neuroinformatik
Word Computer programming Multiplication sign Musical ensemble
Point (geometry) Robot Game controller Point (geometry) Computer Insertion loss Computer scientist Control flow Mereology Focus (optics) Robotics Pattern language Pattern language
Strategy game Hypermedia Robotics Workstation <Musikinstrument> Compilation album Computer programming Musical ensemble Identical particles Dimensional analysis
Robot Asynchronous Transfer Mode Process (computing) Quantum state Different (Kate Ryan album) Personal digital assistant Personal digital assistant Virtual machine Musical ensemble Mereology Virtual machine
Game controller Robotics Virtual machine Process (computing) Musical ensemble Mereology Formal language Virtual machine Formal language
Logical constant Robot Group action Building Projective plane Virtual machine Bit Axonometric projection Process (computing) Robotics Computer programming Universe (mathematics) Videoconferencing Video game Software framework Musical ensemble
Medical imaging Focus (optics) Pixel Theory of relativity Process (computing) Different (Kate Ryan album) Multiplication sign Turtle graphics
Point (geometry) Mobile Web Computer-generated imagery Virtual machine Combinational logic Mereology Medical imaging Telecommunication Robotics Term (mathematics) Different (Kate Ryan album) Videoconferencing Electronic visual display Projective plane Computer Bit Computer scientist Mereology Cartesian coordinate system Computer graphics (computer science) Personal digital assistant Robotics Computer science Figurate number Electronic visual display
Mobile Web Point (geometry) Area Mobile Web Computer-generated imagery Feedback Virtual machine Computer Mereology Medical imaging Process (computing) Telecommunication Robotics Repository (publishing) Computer hardware Robotics Electronic visual display Error message
Area Feedback Visual system Special unitary group Bit Function (mathematics) Medical imaging Process (computing) Robotics Different (Kate Ryan album) Term (mathematics) Computer hardware Autonomic computing Energy level output Video game Arithmetic progression Resultant Position operator Asynchronous Transfer Mode
Logical constant Building Feedback Feedback Visual system Grass (card game) Medical imaging Mechanism design Robotics Network topology Mixed reality Order (biology) output Resultant
Web page Classical physics Context awareness Service (economics) Sweep line algorithm Multiplication sign Virtual machine Hidden Markov model Mereology Neuroinformatik Element (mathematics) Robotics Different (Kate Ryan album) Computer programming Data structure Physical system Algorithm Theory of relativity Sound effect Entire function Virtual machine Performance appraisal Inclusion map Logic output Figurate number Resultant
Point (geometry) Group action Implementation Observational study Online help Bit Line (geometry) Computer Computer graphics (computer science) Element (mathematics) Power (physics) Perspective (visual) Gravitation Self-organization Process (computing) Musical ensemble
Building Line (geometry) Multiplication sign Mathematical singularity Visual system 1 (number) Parameter (computer programming) Perspective (visual) Mathematical structure Element (mathematics) Different (Kate Ryan album) Single-precision floating-point format Vector graphics Energy level Vertex (graph theory) Data structure Normal (geometry) Pixel Form (programming) Area Collaborationism Theory of relativity Point (geometry) Field (computer science) Food energy Series (mathematics) Internet forum Normed vector space Form (programming) Writing
Point (geometry) Rotation Group action Standard deviation Feedback Virtual machine Bit Mereology Distance Field (computer science) Mechanism design Strategy game Autonomic computing Self-organization Representation (politics)
Source code Multiplication sign Feedback Virtual machine Digital signal 2 (number) Mathematics Strategy game Auditory masking Different (Kate Ryan album) Hill differential equation Right angle Simulation
Point (geometry) Robot Digital photography Strategy game Auditory masking Computer programming Multiplication sign Phase transition Hill differential equation Bit
Group action Information overload Virtual machine Similarity (geometry) Heat transfer Element (mathematics) Neuroinformatik Measurement Sound effect Data model Medical imaging Mathematics Energy level Aerodynamics Data structure Domain name Robot Standard deviation Clique-width Bit Transformation (genetics) Visualization (computer graphics) Pressure Bounded variation Electric current
Metre Point (geometry) Game controller Clique-width Multiplication sign Capillary action Virtual machine Sound effect Usability Bit Measurement Category of being Sign (mathematics) Process (computing) Software Robotics Term (mathematics) Iteration Error message
Game controller Clique-width Multiplication sign Feedback Database Line (geometry) Number Uniform resource locator Robotics Personal digital assistant Energy level Procedural programming Figurate number Pressure Resultant
Point (geometry) Multiplication sign Group action Entire function Virtual machine Neuroinformatik Ring (mathematics) Robotics Different (Kate Ryan album) Military operation Order (biology) Finitary relation System programming Computer worm Organic computing Simulation Data structure Formal grammar
Building Gradient Physical law Bit Water vapor Digital signal Thresholding (image processing) Entire function Element (mathematics) Particle system Software Different (Kate Ryan album) Robotics Gravitation File viewer Data structure Simulation Flux Physical system Spacetime
Email Information Different (Kate Ryan album) Robotics Sheaf (mathematics) Materialization (paranormal) Line (geometry) Mereology Simulation
Point (geometry) Particle system Curvature Process (computing) Doubling the cube Distribution (mathematics) View (database) Decision theory File viewer Element (mathematics) Spacetime
Area Machine learning Surface Randomization Temporal logic Building Multiplication sign Projective plane Database Frame problem Perspective (visual) Systementwurf Process (computing) Emulator Robotics Computer hardware Object (grammar)
Surface Mobile app Length Modal logic Virtual machine Electronic program guide Database Code Systementwurf Goodness of fit Mathematics Machine learning Robotics Computer hardware Physical system Predictability Machine learning Collaborationism Building Surface Software developer Moment (mathematics) Projective plane Bit Database Type theory Process (computing) Emulator Computer hardware Mixed reality Universe (mathematics) output Software architecture Bounded variation Resultant
Point (geometry) Robot Arm Surface Video tracking Physical law Moment (mathematics) Virtual machine Digital signal Function (mathematics) Hyperbolic function Control flow Limit (category theory) Element (mathematics) Arithmetic mean Process (computing) Computer hardware Visualization (computer graphics) Robotics Information Bounded variation Organic computing Data structure
Algorithm Scaling (geometry) Information Multiplication sign Projective plane Virtual machine Parameter (computer programming) Entire function Element (mathematics) Category of being Process (computing) Visualization (computer graphics) output Representation (politics) Website Data conversion Data structure
Link (knot theory) Multiplication sign Moment (mathematics) Replication (computing) Replication (computing) Term (mathematics) Robotics Personal digital assistant Telecommunication Universe (mathematics) Robotics Website Information
Functional (mathematics) Boom (sailing) Virtual machine Parameter (computer programming) Mereology Semantics (computer science) Theory Number Inclusion map Goodness of fit Hacker (term) Different (Kate Ryan album) Robotics Color space Energy level Error message Tunis Rhombus Area Standard deviation Information Feedback Moment (mathematics) Digital photography Visualization (computer graphics) Software Personal digital assistant Network topology Resultant Spacetime
Process (computing) Integrated development environment Robotics Different (Kate Ryan album) Multiplication sign Projective plane Combinational logic Computer science Bit Sinc function Number
Integrated development environment Robotics Computer hardware Multiplication sign Projective plane Table (information) Mereology Number Connected space
Programming language Robotics Feedback Self-organization Website Data structure Mereology Perspective (visual) Formal language
Process (computing) Visualization (computer graphics) Forcing (mathematics) Multiplication sign Feedback Virtual machine Planning Cuboid Usability Number Vector potential
Gender Digitizing Projective plane Virtual machine Limit (category theory) Number Mathematics Semiconductor memory Robotics Different (Kate Ryan album) Hypermedia Videoconferencing Representation (politics) Energy level Row (database)
Domain name Presentation of a group Theory of relativity Information overload Temporal logic Multiplication sign Interactive television Materialization (paranormal) Virtual machine Mereology 2 (number) Perspective (visual) Medical imaging Visualization (computer graphics) Robotics Kinematics Energy level Extension (kinesiology)
Functional (mathematics) Information Factory (trading post) Cost curve Virtual machine Materialization (paranormal) Insertion loss Limit (category theory) Neuroinformatik Number Physical system
Observational study Moment (mathematics) Virtual machine Right angle Bit Figurate number Reinforcement learning Social class Template (C++)
Multiplication sign Virtual machine Similarity (geometry) Flow separation Absolute value Element (mathematics)
Sic Civil engineering Cartesian closed category Speech synthesis Musical ensemble Semiconductor memory
[Music] we're about to do our second lecture of the day in this hall it's called trans human expression and it deals with interdisciplinary research in painting and robotics it's at first guided by Professor is not here currently all if Oliver dojin Dwight yes Dawson and here next to me I have the PhD candidate Marvin Guild's off yes and the artist the one who is working with these robots layette kraler yes okay they will discuss both the technical challenges that go into innovation aspects that go along in the development of an e David robot and the suit robot is there since quite some years yet but it's always in development and what I see then somewhere in the future is probably that we can imagine the old-fashioned serious painters of like a Rubens Aram bounder vendek do that at least with filled up with assistants who were very skilled in certain specific aspects but imagine then someone like Andy Warhol and his factory or so that that would be working in a robotised world and I think these people are going to show as or demonstrate us the challenges that are in there because next to that kind of understanding of working with robots as assistants we have to look as well at the social and artistic practice that this way of working actually offers us they do don't go on strike I presume they take sometimes the break and you have to take yeah they care about them are they female that's my first square not masking a name given by Oliver Dyson
it's still masculine okay guys are you
ready for this then I would say light diffuse and put it in play let's go
[Applause] [Music]
[Music]
[Music]
as the robot is recording every brush stroke it has done we can always take those brush strokes and repeat them we can repeat them and shuffle them and use
them in different way it's a variation of repetition of something that is so minor and barely seen that make it I
think to be something that the viewers subconscious will really recognize it
but our consciousness part will not directly see and I think that could be a very very interesting effect language I
can try to play around with this completely abstract painting by the same time as image oriented painting where I have executive same amount of brushstroke exactly the same kind of brush strokes but the each time gonna be put in a different way just the
orientation will be a bit different the location with which one was under and which one is above who will change and this is an idea was that I'm really looking into his took the concept of entropy in painting because the idea of just how house is increased in an organized body so each time that I'm actually making a variation I wanted to increase the level of house either but it still in order to function as a system the painting in the system it has
to still hold some kind of logic behind that and this logic is very hard to control thing from the human mind but you can actually do the computer to help you to create it
we build this painting machine eat a bit almost seven years ago I'm working in
computer graphics for more than twenty years now and the part of computer graphics is what we call non-photorealistic rendering to produce images computer images that look like done by an artist at a certain point I
had idea not just to produce computer images but to draw to build a machine that mimics what an artist does and that
was the beginning of
basically computers right now yeah they used to just follow simple rules and
have completely predictive behavior and right now we're kind of transitioning to
more intelligent systems as we can see in self-driving cars or I read make decision-making in finance for example so they are we're giving more liberty to computers to decide for themselves and we might even be able to advance this more by just going into art where as a human you are I believe supposed to be completely free creatively and trying to transfer that to computing systems and that's kind of the interesting part
because it goes I I think it goes completely against what a computer used to be namely these completely deterministic systems which just do the same thing over and over again [Music] [Music]
however I have no interest to make a painting they're gonna be run directly
only by the robot I actually find very interesting to do something that okay it
done something now and I am reacting
back to it so I will go and I will do it
maybe a glaze on it I will just go and give another very very just roll brush
strokes or splash of paint on it it's
happy that the robot will not do and see
what did we create to the visual sphere
the tog Rafi and monotype the own artistic tools that when they start to be used they created slowly their own medium no no they don't study
[Music]
the Lord not only works as a printer
more like a self supervising a self controlling machine that controls the
painting process itself by analyzing the photos that it takes after some time so
this is a big difference to other painting machines that exist because most of these pyramids work more like
printers because they precompute everything and just print it out yeah our initial idea was that we can reproduce this kind of feedback loop and
we were able to reproduce this using the
machine but now your questions arose what the creative aspects are involved
when humans pain
a small one it's not as powerful and not
not as frightening as the big one
because the big robot it's huge it's got a lot of force and it can't really work
alongside it because if it does the wrong movement you can really get hurt you can just guide it to do something
for you in your painting like yeah fill out this area so you can do something else a parallel
[Music]
[Music]
[Music]
just the question what is creativity is
really interesting because we can't really properly define it and until we can define it we probably won't be able to put it into a computer so in basically the combination of all the real-world experience we have with some kind of you know in our world we have of our imagination to bring that we have to bring both of these into a computer and then there are some kind of process that combines it and yeah it's a more exists as a human you know it exists as I'm
struggling you can't really put it into words or even a program
[Music]
[Music]
for a year time I was studying some nice calligraphy painting then every something that I learned about it it was about the embodiment of the act it's about how you move your body and the use
of ink on rice paper it's something that you can barely control you always create the same kind of a pattern of how a flower should look like that you cannot control exactly how the ink will spread
on the rice paper you cannot erase or change it the thing that you need to focus on is how you move your body it's
not how you perceive what you see and I think one of the most interesting point for us was this discussion about control
and loss of control in the act of making a painting as for the computer scientist
the most important part is to control it to predict everything that the robot is
doing
most people that come here really think that we want to reproduce other artists
but that's not really what you want to
do we want to explore how an artist in general work are the certain strategies that you can have for certain artists and not really to copy artists
in station women we should select Commodus next some reason we have a dog for but these are focused on tourism yeah no it's not from toboggan that was difficult on my dimension as well it seems to help us in the media
[Music] by actually programming a robot to paint
which you but not for you I think it's a very important distinguish for me Atlas
using the robot as at all as a painterly tool it's something that helped me to understand where my conscious is part of it where is my individuality is part of
it and where it's about the execution of different kind of material [Music] Wow currently most of the cases to machine
works in fully automatic mode and and we are fascinated by that yeah you your your start with something on the next morning after like 1012 hours of work you get an interesting result in in some cases but my final goal would not be
intellect a machine pain by itself instead I would really like to work together with the machine so my dream would be like in an artistic process I
want to teach the machine in a very
efficient way but I want to have on my on my canvas i I had to learn that the best thing if an artist comes is to let
the artist find its way mr. machine
and as there is nothing remaining
robotic paintings out there today it's
very hard for me to say okay this isn't the tools what you do it I'm still trying to understand that so you're part
of it is to really to learn to let go of control another machine first the world to take me by the hand and tell me okay this is what I can do let's go for so I
was do dis and I'm gonna react to it and we slowly start to achieve I would like to believe a new language [Music]
Oh [Music] Hey
hello thank you very much for coming and we had this video so you could kind of
see the robot in action it's a bit difficult to actually get the machine here to such a place to demonstrated life quick a quick introduction my name is Marvin gold so I am now a PhD candidate at the University of constants and my job mostly is to actually build the robot program it's okay not built we buy industrial robots as you have seen and then built the painting set up and I kind of provide the framework where either the machine can paint autonomously or we can have an artist with us who works with the machine I inherited in quotes the project in 2018 Tomas Linda Mayer who you saw in the video he did most of the work up until now and I'm kicking it up now modest and
practice image making in relation to the pasta turtle age we living today namely we overrun by so much imagery and what is the difference and why painting is in
hardly different from actually seeing an image digitally why's that pixels and so on then his worst not collaborating with the University of consensus 2016 and doing this exactly time and the work infighting makes the subject matter of the entire work to be the process so this is right now in front of the focal point of my work now quickly to go into
the technical part so how the machine works more or less firstly the name it's
a backronym for electronic drawing a parrot or vivid image display so obviously we wanted to call it e de vide and then found something matching we started at ten years ago and it was really just that robot in the basement it had some wooden canvas in front of it where we tried to draw in some way we
started and felt it we moved over to using ink brushes like Japanese calligraphy brushes and finally kind of graduated to regular painting with a paint on different kind of canvases and different kind of brushes and we're doing it at the University of Constance as we said in the department of computer graphics because it's it's a bit difficult to find a home for such a project if we did only robotics painting would be a bit you know he adds a nice application but not not the point what we're getting at and computer graphics is a bit of a sweet spots the goals we want to achieve with this project are many years so one is autonomous paintings ideally we'd like to have to the machine in the room close the door give it some instructions then it produces a painting by itself we can pick it up and it's done that would be one aspect but also to provide some kind of into a tool for the artists so either as an assistant as mentioned by The Herald so we could have an artist who just makes a sketch and the robot fills in all the tedious detail works or the artist can do something else in the meantime or as leah is doing figure out completely different use we did not think about for this machine because as computer scientists we think as mentioned in the video in terms of controlling this machine house it would find it at work and not necessarily explore all the artistic possibilities this project is a bit of a combination between engineering robotics computer science art and cleaning because as you also see in the in the background down there having these robotics and paint
which dries all over the place can get a bit annoying and cause some problems we also have a mobile demonstrator for exhibitions not the small yellow one you saw we got rid of that one we now have a about this tall robot by ABB which we
sometimes bring to exhibitions it's still a lot of work and but maybe in the future we can show it off somewhere nearby how the entire machine works is
like this we've got our canvas where all of the painting takes place the robot knows it as its workpiece and then by moving along certain points in the image and having a brush in its kirpa it does produce some brushstrokes on the left in
the yellow and green highlighted area you can see our paint repository which is just a pot where the brush is held into it in some interval picks up some paints discards the excess and sometimes you also clean it and that's just a water jet where the brush gets where all the oil paint gets removed and we are ready to move to over to another paint the entire process is supervised by the camera you see in the top left in blue that's kind of what sets us apart from other painting robots that might exist we have a feedback mechanism which can supervise and adapt to some errors that
might occur so here in the top left you
can see what the robot in autonomous mode would start out with it's just a normal JPEG he would hand it to the robot and it would compute some kind of paint recipe and say I want seven colors mixed in in these ratios from some basic paints I put it in these slots and then it can start painting initially it just places some stroke to try to get certain
areas to match the input image so the blue sky it will deposit a lot of blue strokes the yellows sunrise area it will take its yellow paint and put it over there the initial results of course are not very good because we're trying to get ahead quickly we're using a big brush and but after we deep position some initial paint we create the difference image which you can see in the top right and in right the areas the the robot considers is more or less complete so it does it can't do much more to improve this area in terms of getting closer to the input image but the green areas there it can expect some more progress so for example the Sun the yellow area is not bright enough so it's marked in green while the beach in the front and it's dark ish enough for this level of detail so it just skips that area for now until it maybe moves on to a smaller brush based on that we compute new strokes which are a bit difficult to see but this is our life debug output which we can see while it's painting which are then stored and executed in the painting process and after repeating this many many times like over one day - sometimes even three days we get a result like that on the left
which is kind of close to the input image and we can definitely say we have reproduced that image in some way our early attempts using only the ink brush
looked like this and we already we see some mixing like long strokes for the tree and short strokes for the grass below it this of course was done by masking the robot doesn't really know what it's painting it just follows our
orders in this case and after as I said
after the inks we moved on to acrylic and these are kind of our flagship examples the best results we've got so far and building in the hub of constants and a car in the desert and you can see the you can see the details which the robot has managed to achieve using its feedback mechanism you can also see some problems we still have like the antenna is a really small detail and our current mechanism it's not perfect there are many issues in which it bumps into and I also notice it's kind of an art to
figure out what kind of input is good for the robots so which weather has it doesn't have a chance to produce a good result because we do have a bunch of stuff we would we don't show we kind of got rid of and speaking off stuff we got rid off okay so there was a kind of as a
side effect happening while the robot is operating in automatic motors and beside the main painting it's actually creating Louise an extra paper and on this paper the rabbit is being program just to sweep extra paint that was being docked first the brush was going to taking
paints taking white much so it's going to interact the small paper to make those side kind of a small brush strokes and I find these actually paper to be much more interesting than the painting itself it was being produced by the system as someone who's coming from a different kind of a background of painterly background I always look to those painting and thinking why would you like to do this services this interest of there is a technical challenge but there is as well the reason of why we do what we do and I am coming from like I actually got educated in in painting department here in Leipzig it's a very classical Academy for painting in traditional European manner and when I graduate I remember like standing there and looking around thinking hmm it's all good but where I am coming into this entire discourse I'm coming originally from the Middle East I have a different kind of a background cultural background that makes you think why do we adapt different kind of certain love aesthetics and how we work around it and one of the major question that I came to understand my exploration is that what for me will make any painting right now the most interesting is looking into different structures that shows logic in them rather to be at the coil stage more interesting than into going into narrative painting namely things with let's say semantic evaluation which is a thing for the computer part is well very interesting if the machine how can you make the machine aware off breaking the painting inter- elements this is the sky this is a house this is a bird how they can actually being taken apart together so looking at those works I start actually taking those pages and I interact with them I added some more unconscious careful brushstrokes to go together or like more gestural while to go on this like very structured one coming from the algorithm of the system at the same time the machine the paint is dripping there is some kind of irregularity so it's always about this small relation between things that have
a highly personal organization and things that are a bit less so if I would kind of try to think of a metaphor of it it's like taking the painting realm and to track it as a as a rubber that you you kind of pull as much as you can till the point that you know if I will just stop pulling it right now it have the strongest like gravitation to to each other back or it could actually snap and ripped and this is the point I'm kind of investigating with the help of working with a computer graphic
Department so this is just kind of implementation that I start to understand that to actually before you even go and start talking about painting one need to talk about the first thing that what painting is about and this is about brush strokes it's about lines and this is the first thing of whirring of movement and we have today or not today for many years already the use of computational aesthetic and power in art especially in music however there is something about the sound that it's being done it's gone and painting allows you to actually record every movement to record every action and to go above it and to manipulate on it and I found these elements to be specifically interesting and that was the base of the
creation of some of my earliest work I've done 2016 in collaboration with homicidal maya at the time at the University of Constance where I took different kind of parameters that goes very basic mathematical structures and to see how one can build different kind of all-over composition a lot of these works actually be done and the gif style psychology writings and I was just
trying to take those ideas and translate them into basic mathematical equation and execute them into painting the other level that I start to investigate is the
relation of a single brush stroke that you could have seen in the ones before in relation to actually how many perspective we need to actually create a form and how that is being put in a compositional area so it's a very technical questions but at the same time those structures are the one that created theater of perceptions you have one an element when the individual
strokes become as part of pain so when the one become a group and how do we like we understand when we get to the point that we understand how we make the machine to understand those situation when to use different strategies when making fainting back to be integrated individual feedback more like autonomous painting methods another important part that we worked on is how we can make the
strokes to look more organic so if you look in the right lo you can see that the strokes are very straight they're varied they look more mechanically done so it how can we introduce things that just a small deviation of the strokes or like the brush not to go like 900
percent like 90 percent but a bit like this and a bit like rotation and saying we want to make it as part of paint here but don't make it directly some things to go a bit out and you know all those like small details is something that for me are very important in creating something that we as people can relate to and feel this is actually a work of art and not something mechanical and distance from us is by increasing the fields of of organic representation so this is like one of
the the work play around the tail I've done with Thomas in the beginning of my residency showing the work on the right is done 100% by the visual feedback well the second was being done according to breaking down the entire concept of the painting into masks and using only three colors so both of them are being done
with the machine just using different kind of strategies extracted from this picture of these lovely guys that he does not even know that he's being shown here right now no idea who is actually lost the time yeah and that developed to
the work the three of kind of funny idea let's make of course what is like when you acknowledge a painter like A to B as an entity then this is a question one can take as well into the art world discourse the commercial art world discourse but it was the classic self-portrait of the painter and this is
a kind of the preparation phases of how we done it and that was going again into the strategy of actually breaking down the painting into masks and giving different strategies of how we do those paintings so we actually based this entire work on a photograph that then I worked out a bit with GIMP and using the custom made program that Thomas they built for me I created different kind of masking strategies of brush strokes to create the painting that this is just the self-portrait instead we're trying
to be very hilarious he even signed his name and going back like after of that
all of those work is still her to this point of what makes a painting more interesting why we still interested in and why in this time that we overrun and
overload with with visual imagery painting is still important people still going to learn fighting we still do that and I'm always like tricking between the elements of making and the of the perception of how we see it and that was a kind of this work was built on the idea of like how would I say like the action of making a structure of just a stroke it's very doodle looking structure that was done by the computer and something that it's still complex enough for human being not able to repeat only a machine can do it but the deviation of the changes in the variation of the work is actually been consumed or created through the use of paint so how often the rubber takes paint more or less and then just make her feel old exactly it feels like it was being painted quicker but it was actually a bit drier the brush so how one can make those like small manipulation and tricks for the creation of painting only the domain of a brushstroke what we have currently as
you can see we are kind of painting in all ways similar style we have got a brush we apply it in the at the same pressure level to a canvas then we get some paint transfer and that's how we create our images and because it's very
difficult for our robots to actually handle a brush we don't have it will Centrepointe which everything robot rods running inside these machines usually you have a welding gun you know exactly to the millimeter here the effect of metal will be localized but brushes they deform not transform that shouldn't happen they get dirty it's really difficult to simulate a brush because of these many hairs they vary their properties when they get wet and so on and and the provided software we have for handling tools and robots only works with these hard hard tools basically not soft tools like brushes and that's a problem for our painting and also what we can provide to the artists as a control element in here in the painting process because for example it's difficult to define a certain edge in a painting the painting process will draw over it many times and since the brush deviates a bit every time and the corner will just blur out hence we can too many details and we don't really know the side effects the movement generates so if you take your brush move it and then perform a sharp corner it will slide around on the under canvas a bit and produce a certain effect we wicked can't control for that yet so our solution for a stopgap solution in the first iteration is to actually measure measure what our tools can do in terms of width and lagging behind so if you look at a brush like this and we we
think we have a nice defined tool center point but as soon as you apply it obviously it blurs out like that it deforms it might stay like that and as a funny side note if you overdo it with the robot and have some sign errors you deform it permanently so our solution to this problem is doing this we have some
automatic procedure I'm not going to go into too much detail but basically the robot can paint strokes at non pressure levels which we limits to what what I'll show you previously and then we can use our feedback mechanism to measure how broad a certain stroke gets at which pressure pressure level and using this we can accurately reproduce strokes for
example in this case we can account for width we can also if a human paints the stroke you see on your left with the red centreline we can record it using our feedback mechanism and paint some first attempt like we do normally with the yellow center line then we try to figure
out what went wrong when painting this where a brush might be lagging behind where we're not applying too much enough pressure and so on we compute some correction and can perform the stroke again this happens over several iterations and in the end we can see a quite nice end result but here you see there's already a big improvement between our initial attempts and what we can actually produce and by storing these strokes we can kind of build up a database of knowledge over time which then artists can use to tell the robot hey put stroke number 25 in this exact location thus enabling much finer grained control and kind of as a party trick by just reproducing single strokes
and what writing in a funny way we can also make the robot just repeat you're right it's like a medieval monk if you're so wealthy yes then single strokes so as
you can see like our work has changed the material before the work was been done lots with acrylic painting means that you can work in the ring and in order to really control and to master
the way of you doing the brushstrokes we decided to move back into a working more with ink and in paper where we can actually have the ability to see every stroke into its details and to understand how we can control it better how can we pursue better what we want to achieve this is just an example to see
the entire kind of spread of a stroke using different kind of of points it's a saying okay we have just the time in the the second one just the time how long it is attending so just to see how the material is reacting so we are talking here about not only the computer itself and the stroke but actually the material
itself the paint is reacting if the brush is a bit too wet it's actually expand more into the paper a bit less or a bit more like the gradients of the color is changing according to how often is actually dip into paint so in this situation it was dipped every 80 strokes in paint but every 10 strokes in water so the water is well that is slowly diluted and and the paint got as well they'll read in there was kind of situation of radiance that happened through those kind of a system that goes to us again Indiana's viewers and together for this work Marvin had here
had built this software for me that generated particles and those particles are actually being created due to Newton's law of gravity I just decided to use this as a general law of how we can spread structure how things are operating in space if I'm looking in the painting as a space of creation like in the world and we can scale it into different situations and different sizes take all the elements from it and I use actually this work to create an entire installation build from many elements
into xxe different parts so again this entire work was being done using ink in the rice paper here just to show you again the differences of how just using a different brush make the outcome to be definitely just totally different
although it's the same sections the same information that was being given into the robot for painting the fact that the
robot is not aware of the material is working with a lot of situation happens where the material is getting this like really deformed what probably as a trained artist I would think it's maybe wrong or maybe I should not do it or we try to avoid it and the robot is not aware of those situation methods actually allow us a new way to handle
with materials to say oh it's actually an interesting situation to have those holes happening in the papers like what happened with too too much going over the line so those work were being treated
thinking 20 something rice paper work that I paint had taken from this particle generator and been hanged in a room in a insulative way it's giving us the point again to observe painting not as one big thing that you have to look seem intelligent but actually forcing the viewer to walk by it and to stop and
to look about every little elements and to understand and discover the process of making it so making the process much more visible this work again was been
later developed into how one can exhibit the work so to go outside of the painting from just things as it's flat on the wall and to take it out into the space so therefore there was a decision of framing those works in double science
using hanging frames and you can actually look it and see them from the side it was being painted assignment was being just absorbed and creating a special area where one can work and experience the process of the creation and not only the finished object as a painting
today and but the project is of course going to continue and be developed further over time one major area from my side I want to work on is painting technique so right now we've got one style we can do well but of course it would be better if we could more emulate what the human actually does the human does not stand in front of his canvas and play stroke after stroke in random places we would like to make the robot
scribble on the surface mix some strokes deliberately may be performed apps and vary that style within one painting may be as is appropriate and of course getting a machine to actually do that requires a lot of input of certain movements and it's a bit tricky also I mentioned the database of movements would like to build so we know ok this movement will give us this exact results we have a much more predictable process and something you can pick and choose from and maybe even tell the robot ok only focus on this type of stroke instead of this so you can in the end guide its style in a much more concrete way than right now where you can only vary stroke lengths and stuff like that also our system design you saw a bit of the development we started out using a plastic bucket with some red in it to clean the clean the brush and we're moving on prototyping as we go on and trying to develop better hardware so the robot can actually have clean tools save to changes and so on also open necessity is a problem at the moment and it's a bit difficult to get other codes into our project or to just get get strokes from somebody else you would like to have executed on the machine so we're reworking the software architecture to make to make it possible for other collaborations because it turns out we've got competition in some way in a good way in Turin I think there's also a painting robot being developed at the University we are cooperating with Shenzhen University will I can kind of reproducing our robot and then we have two of these systems to work with in different places also because I have to mention it machine learning and we need to figure out if it
makes sense to actually use machine learning in our approach if there is some is way to do it and is it possible to maybe simulate some creative aspects you know these Google deep reme thing is could we
take that technology get all some strokes and have have it output on our machine and the interesting sidenote who actually gets copyright if we allow to do the mission if we allow the machine to do more and more by itself so at some point we had a law professor in our lab and he asked how much are we still doing in this process because once the machine starts doing enough and reaches what in in German law is called chaplains who I think it will get difficult to attribute to who actually owns the painting which is being created and some future issues for yet ok so the work that we're being
done still working on stopped for me right now in the moment of how can I use this tool to extend myself it means how can I go beyond the limitation of myself as a human mean what I can do first of all for my own perception because when I'm standing in front of a canvas a canvas or a painting surface I'm looking from one place my brush strokes ability is only limited to my sides of my arms so I would like to integrate more into those kind of elements to see how can we
use this tool to create maybe a larger scale works at the same time idea of how one can translate data taken from different elements into the painting so the painting will become an abstract representation of maybe social structure if we take it from movement data that I'm right now collaborating with the casa para Nene in general taking data from behavioral movement information and using individual human beings actually to be translated into rush strokes further it's the idea of how the machine could become more and more organic in its behavior no so it's not only the way it looks by the way it works and operate so right now it works very deterministic ways like if you give it a JPEG it works from it according to the parameter it's been given or it's giving through a deterministic way of an algorithm and being given I've just been filling it up I would be very interesting to get it point where the machine will integrate what considered to be a mistake in the painting process something that we didn't expect it to happen we didn't want it to happen but it's there now and it's changed the entire world situation so how can we integrate and react to each other so they converge the conversation that happening between me and the machine is actually therefore opening in you around for creativity because there is something almost objective that comes from this kind of an input that makes me question myself of my decision-making process and my habits of visual information yeah that's
kind of what we have to say so far if you want to figure out more about the project we've got a website from the
University if you want to see more of Leeds artwork she has got a website as well and what we presented today more or less the stroke things you can see in our let's publication at mdpi self-improving robotic brushstroke replication a professional would have added a link here but we added that last minute I'm sorry but you can find it easily I have to use the term we can easily Google it sorry yeah that's our talk so far if you've got any questions I guess now's the time
great I'm actually wondering how you make a robot really though at a
certain moment you have something like a come on let me get into this because isn't it because that kind of communication would be interesting with an assistant you just kick it and he or she goes away but II David it could kill you if you actually that's experience we
have in our hacker space as well we have a diamond gothic thing in there boom drops sometimes whatever we have lots of questions I assume really start number
two please hey thanks for the talk if I understood correctly it's not a printer because you get more because you get visual feedback yes okay so I'm
wondering what how does it actually can you scale beyond just visual feedback I mean there are a lot of different ways but is it feasible and that what would be the consequences because it could be you know another social feedback by for example us or the public of the audience judging the result I made notes so many ways so how is it was it how is it feasible practically speaking what are the consequences so the question is can we introduce some other feedback other than just with visual information yeah how you live all more conceptual network yes you could so in theory we don't have it at the moment obviously but if we had a way for somebody to kind of give the robot an overlay and say this corner isn't good it's kind of deviate from my optimum by a certain value then we are we would also need some function to determine how it should improve that and then we could also include other kinds of feedback of any kind basically the trick is always to find a function which describes how big the error is for the feedback and what can be done to fix it in cases of just black or white or color a color spaces it's kind of easy if it if it's too gray you paint more white color over it but in here it gets difficult but it might be doable or do you have any well I think it's there was a lot of challenges to use of the visual feedback because there is the question always of perception when the visual feedback that what is actually getting right now it's from a photograph where everything is being scaled equally and there is certain kind of parameters it's checking if what Mommy just said about the darkness the level of areas and so on and it still does not have the tuning of understand for example semantic situation to say oh here I painted the the tree as part of the the sky and it's supposed to be two different things this is a thing the interesting part for us as human being to interact and slowly teach the machine and see what it's doing and correct it and hopefully the accelerator machine will remember through its doing to get to a more defined or abilities of feedbacks that are probably much more developed from what we have right now I would love to see a live demonstration here number one
please truth thank you for this very inspirational talk I actually got
several questions but I will limit myself to you if it's one one one one so first question is could you please elaborate a little bit more on this aspect of cleaning since you said it's a
combination of art robotics and cleaning then question number two would be about
yeah what what would for each of what was for each of you like the most surprising in this project and cooperation what what's all this about cleaning I mentioned basically I mentioned so much because it's my major headache in this automation process right now over time when the robot paints a lot with that brush some stuff dries some stuff just it sticks in the brush and it's really difficult to get out then you continue continue continue at your other paints and everything tends to add some gray brownish stickiness I think is well it's the differences of how computer science are used to work in certain kind of a lab and suddenly they have an atelier happening in there yes so when I was there for me it was like actually it's actually it's a fairly clean studio here so I think it's a lot of matter of you of habits but of course when you work with with machinery and you work in in a scientifical environment you would need to measure everything so keeping it clean environment is important to actually follow about the results and that I think one of the reason why it's such a big headache for AMI wouldn't it be better to come over to the athlete and with your robot since that's the
atmosphere and environment two different cities that's too far away from each other yeah we try that was one and also another problem is if you see the table
with colors on the left there's also a magazine where the robot can swap out brushes and once too much paint had accumulated in there and the tool did not properly enter the slot it was misaligned the robot then almost threw over the table when trying to switch tools yeah so that it's also just a mechanical problem after some time the floor doesn't look like an apple yeah
here no I think yeah when you're there
you see a lot more okay and the next question was should we do the second one or what was the most surprising part I am surprised I'd be quick when I entered the project I was like do we have a robot yes can I walk with it yes do you want to know what we do no and then I figured out oh this art part exists I had no no connection with it at all and that was kind of surprising for me to believe it stumble into this world which which also is included in the project we can imagine number two please oh yes oh
yeah the surprise yeah I think for my
site it's very similar to from maybe my Marvel parts because when I came out I thought okay it's gonna be where it's coming from very different worlds how can we discuss very similar issues and it was a pleasant surprise to discover that we have similar questions and we look into the world in two very similar ways but the tools that we are doing it in perspective Detroit is very different so the attempt of trying to communicate those two is for me the most interesting part and just looking into structure to understand how can I use computer language structural language to actually build painting and how similar and close to each other those two are organizing cares or gay lysing organizations
something like that entropy yet please sir I can can the robot make use of some tactile feedback
for example potential sensor in the brush we don't have it and it's very difficult because we the forces that should be occurring are very small and might also change because of stuff accumulating and also as soon as we introduce a wire into a brush for example to measure something that will change the outcome so no but we are thinking about maybe having some live supervision of the brush with the camera and but then you get the problem of a green brush on a green background and trying to separate that it's it's difficult so we're still painting more less blind and that if you have a solution come up and yes and it over number two what number one please as an artist experimentation is very important to try and experiment experimentation is very important to try out new things to you know do things that you maybe didn't plan to do and so and so on what happens if you David gets a brush that it doesn't know if there's a tool in the in the tool box that it picks up and then just applies it to the canvas I actually done that with it several times and it just makes the it just make it keep painting if especially when you work with the visual feedback because things happen that it didn't expect it to happen so the Machine thing it's wrong I will just do it again and
it could keep probably paint like this for eternity because it doesn't have this like you know experimentation something that you can tell to a human because then I experiment something new and I understand something from the process and I'm learning from it the machine is still not in the situation
that it's learn it has a goal it has its tools and it's no how can I use those tools to achieve my goal it doesn't understand swearing or something like that no but sometimes I do swear it depends we have a couple of more minutes
memory because the Machine does not have
much sensory feedback but if we do it in some controlled way well it knows limits so it must be constrained in that way we could definitely implement methods where just paints randomly records it and then somehow uses it in some further experiment yeah number two should shoot yeah thank you very much I have two more of art historical questions first the notion of the painter right the notion of the artist as painter is a notion that is like marked with a lot of markers of gender for example like the men that paints but also of virtuosity of authorship and things like this have you think about like dealing with robots with other media that is marked this way or maybe with painting but differently or how can one change this notion that kind of gets repeated and also I was wondering yes I was wondering about people like John ting early like people that were dealing with machines already I don't know in the 50s or something but they understand themselves of sculptures too like they were producing a machine that was mechanical and that was painting like a painting machine and of course what you're doing is differently since it's a robot but how like if you relate to that tradition thank you so for the first question I am a painter and it's important for me to stay that although according to my Academy aware I studied they consider it not to be a painting anymore because it's not exactly done by the authorship of a human being although I'm working with paint and brushes I was being sent directly into a video audio department when I wanted to produce to show this project there's like a there is no audio video in the work why do you sing me there so when it comes to using different media for example now the project I was doing in Italy was working with using digital representation and different levels of
visualization that I found interesting but to be honest for me the interaction between materials is the most important part and using the machinery right now is an extension of how one can use technology for its own creativity so how we retain look so I try to see it in the relation of art history as an opportunity to establish a new aesthetic and not to go into the discussion of
authorship although it needs to be there and it has to be there because of course when the camera was introduced photography introduced like in the beginning and consider not to be even art and it was very low valued because the Machine down is not the person today the discussion is not there anymore so I think there is this freedom that I have right now and its domain because I don't have the ultimate stars to sit and look and upon and to say I have to imitate those like all European matter the way they've been painted because it's neon so I can do whatever I want and no one can tell me it's right or wrong and this is a great privilege for the second question about the way of presentation I would still go back to the say I am a painter and for that I'm interesting in the image itself and I think there is the fattish ISM of the machine especially today when we see the robots and people don't even care what it's doing a see Roberts moving it's like oh it's so cool and it's nice and I think you can do a lot of things in levels of interaction works and but I am really not into this like it I've not in another performative artists I'm not interested if making a kinetic art I'm more into really just how can you represent the movement in the steel in the still image and how that could in a way fight back against the overload of visual imagery we have today because we maybe look at the painting for three seconds we get it we go on like it's a visual artist today I've been required to know who are my contemporary fellows I cannot be at the same time in Hong Kong and in New York and in Paris so I check them out in my shitty phone and I see this image everything I get what it's done there and I'm continuing so
it's a different way of experiencing information and I want to use this machine as a way as a tool to communicate back what the information coming from the machine from the computer that we are all using right now and reconnect and we communicate with real materials again
yeah almost in the factory of Warhol but number one please we have just a couple of minutes left and I'm very curious about the machine learning side of things so of course there you would need
to define something like a cost function or a loss function which describes how well the system is progressing and I'm wondering both artistically and scientifically how would you go about what are you thinking about defining such cost or loss or quality functions that's what exactly what I'm working on right now and what's giving me nightmares because it is very very difficult you try to reduce the problem and then you already impose extreme limits on the machine so what maybe what
would maybe be a good first step because at the moment we do no learning at all in the current setup would be reinforcement learning and just have the Machine approximate some kind of template and maybe afterwards we could say certain class of stroke is aesthetic in some way but it gets very red vague and difficult so we'll we'll figure that out in the future I hope also I guess artistically it's a very interesting question right have you talked about how you define those questions this is a always a quick what is wrong and what is
right in the arts what so it's it's a ratio if we find something I think it would be troubling if you could say yeah this old painting is 85% aesthetic yeah
but I think there is today a huge amount of studies where that I found a lack in our science that I found a bit scary to
be honest the trying to quantify those those elements and one cannot look at it as an absolute because it's all it depends on the place and the time and the history that you're coming from and this is a you challenge that we are facing today as we all even it's a global society and everything is flattening out and everything has been done and being taught in art academies across the clubs are kind of similar so I find it to be very dangerous and this
is why for me this working with the Machine is about given autonomy but using it as a tool to maybe even like learn its individual user so maybe we can create several kind of users and each user learn the machine will learn the habits of its user and therefore we'd continue into a relative absolute values I would say I really have to apologize because you have to close this session I'm
looking forward to these new chapters of artistic behavior in humanity I suggest to continue these speeches afterwards somewhere here in a car in a bar or yeah you will be available here I would love
to tank Li out Marvin II David of course and professor Dawson that's the name
isn't it please thank you [Applause] [Music] [Music]
Feedback