Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/1093 Thanks!
[Music]
we're about to do our second lecture of
the day in this
Hall um it's called transhuman
expression and it uh deals with
interdisciplinary research in painting
and
Robotics it's um at first Guided by
Professor who is not here currently
Olive Oliver Doan do I yes Doan and here
next to me I have uh the PHD candidate
Marvin gildo yes and the artist the one
who is working with these robots leat
ker
yes okay um they will discuss both both
uh the technical challenges that go and
The Innovation aspects that go along in
the development of an e David robot I
understood the robot is there since
quite some Years yet but it's always in
development and what I see then
somewhere in the future is probably that
we can imagine the old-fashioned serious
painters of lier rubben or rambon or van
uh who had aters with filled up with
assistant who were very skilled in
certain specific uh aspects but imagine
then someone like Andy Warhol and his
Factory or so that that would be working
in a robotized world and I think these
people are going to show us or
demonstrate us the challenges that are
in there because next to that kind of uh
understanding of working with robots as
assistant we have to look as well at the
social and artistic practice um that
this way of working actually offers us
um they do not go on strike I presume
they take sometimes a break and you have
to take yeah take care about them are
they female that's my first question no
I think David is really a masculine name
given by Oliver Dyson so yeah it's still
masculine okay guys are you ready for
this uh then I would say light the fuse
and put it in Play Let's Go
[Music]
w
[Music]
as the robot is recording every brush
stroke it has
done we can always take those brush
strokes and repeat them we can repeat
them and shuffle them and use them in
different way it's a variation of
repetition of something that is so minor
and barely seen that make it I think to
be something that the viewer
subconscious will will recognize it but
our conscious not part will not directly
see
[Music]
it and I think that could be a very very
interesting um effect to play
with I can try to play around with this
completely abstruct painting but at the
same time as image oriented painting
where I have exactly the same amount of
brush stroke exactly the same kind of
brush strokes but they each time going
to be put in a different way just the
orientation will be a bit different uh
the location with which one was under
and which one is above will change and
this is an idea that I was um that I'm
really looking into is the concept of
entropy in
painting because the idea of just how C
is increased in an organized body so
each time that I'm actually making a
variation I wanted to increase the level
of of Chaos in it but still in order to
function as a system the painting as a
system it has to still
hold some kind of logic behind it and
this logic is very hard to control a
thing from the human mind but you can
actually use a computer to help you to
create
it
we built this painting machine e David
almost 7 years ago I'm I'm working in
computer graphics for more than 20 years
now and the part of computer Graphics is
what we call non phistic
rendering to produce images computer
images that look like uh done by an
artist at a certain point I had the idea
not just to produce computer images but
to draw to build a machine that uh
mimics what an artist
does and that was the the beginning of
eay basically computers right now they
used to just follow simple rules and
have completely predictive Behavior and
right now we're kind of transitioning to
more intelligent systems as we can see
in self-driving cars or algorithmic
decision- making in finance for example
so um they we're giving more Liberty to
computers to uh decide for themselves
and we we might even be able to advance
this more by just going into art where
as a human you're I believe supposed to
be completely free creatively and trying
to transfer that to Computing systems
and that's kind of the interesting part
because it goes I I think it goes
completely against what a computer used
to be namely these completely
deterministic systems which just do the
same thing over and over
[Music]
again
however I have no interest to make a
painting that going to be run direct
only by the robot I actually find very
interesting to do something that okay it
done something now and I am reacting
back to it so I will go and I will do
maybe a glaze on it I will just go and
give another a very very gestal U brush
Strokes or splash of paint on it
something that the Robert will not do
and see what it will create to the
visual
sphere theography and monotype they all
artistic tools that when they start to
be used they create it slowly their own
medium their
own own
[Music]
aesthetic the robot not only works as a
printer more like um a self- supervising
or self-controlling machine that
controls the painting process Itself by
analyzing photos that it takes after
some time so this is a big difference to
other painting machines that exist
because most of these painting machines
work more like printers because they
pre-compute everything and just print it
out yeah our initial idea was that we
can reproduce this kind of feedback loop
we were able to reproduce this using the
machine but now newer questions um arose
what the creative aspects are involved
when humans
pain the small one it's not as powerful
and not not as frightening as the big
one because the big robot it's huge it's
got a lot of force and you can't really
work alongside it because if if it does
the wrong movement you can't really get
hurt you can just guide it to do
something for you in your painting like
here fill out this area so uh you can do
something else in
parallel
[Music]
all
[Music]
just the question what is
creativity uh is really interesting
because we can't really properly Define
it and until we can Define it we
probably won't be able to put it into a
computer so it basically the combination
of all the real world experience we have
with some kind of an you know inner
world we have of our imagination to
bring that we have to bring both of
these into a computer and then there are
some kind of process that combines it
and yeah it it somehow exists as a human
you know it exists as I'm struggling you
can't really put it into words or even a
program
[Music]
a
[Music]
for a year time I was studying some pese
colography painting and there is
something that I learned about that it
was about the embodiment of the ACT it's
about how you move your body and the use
of Ink on rice paper it's something that
you can barely control you always create
the same kind of a pattern of how a
flower should look like but you cannot
control exactly how the ink will spread
on on the rice paper and you cannot
erasee or change it the thing that you
need to focus on is how you move your
body it's not how you perceive what you
see and I think one of the most
interesting uh point for us was this
this discussion about control and loss
of control in the act of making a
painting as for the computer scientist
the most important part is to control
and to predict every little thing that
the robot is
doing most people that come here really
think that we want to reproduce other
artists but that's not really what we
want to do we want to explore how
artists in general work are there
certain strategies that you can have for
certain
artists and not really to copy
[Music]
artists next we
have Focus
[Music]
by actually programming a robot to paint
with you and not for you I think it's a
very important uh distinguish for me at
least um using the Rober as as a tool as
a pain tool it's something that helps me
to understand where my conscience is
part of it where is my individuality is
part of it and where it's about the
execution of different kind of uh
[Music]
[Laughter]
material currently in most of the cases
the Machine Works in fully automatic
mode and and we are fascinated by that
yeah you you um you start with something
on the next morning after like 10 12
hours of work um you get an interesting
result uh in in some cases but my final
goal would not be to let the machine
paint by itself uh instead I I would
really like to work together with the
machine so my my dream would be like in
an artistic process I want to to teach
the machine in a very efficient way but
I want uh to have uh on my on my canvas
so I I had to learn that the the best
thing if artist comes is um to let the
artist find its way with the
machine and as there was not many um
robotic paintings out there today it's
very hard for me to say okay this is the
tool is what you do with it I'm still
trying to understand that um so youre
part of it is to really to learn to let
go of control and let the machine first
of all to take me by the hand and tell
me okay this is what I can do go cool so
I will do this and I will react to it
and slowly start to
achieve I would like to believe in new
[Music]
language
okay hello thank you very much for
coming um uh we had this video so you
could kind of see the robot in action
it's a bit difficult to actually get the
machine uh here to such a place to
demonstrated live um quick a quick
introduction um my name is Marvin gulo I
now a PhD candidate at the University of
constant and my job mostly is to
actually build the robot program it okay
not build it we buy industrial robots as
you have seen and then build the
painting setup um and kind of provide a
framework where either the machine can
paint autonomously or we can have an
artist uh with us who works with the
machine I inherited uh in quotes the
project in 2018 toas Lind who you saw in
the video he did most of the work up
until now and I'm picking it up now more
or less
and Hi H thank you again everybody for
coming my name is D I'm a visual artist
and over the last years I'm been
focusing on how we can actually work and
practice image making in relation to the
posst digital age we're living today
namely we overrun by so much imagery and
what is the difference and why uh
painting is intirely different from um
actually seen an image digitally wise
pixels and so on and is where start
collaborating with the University of
constant since
2016 um and doing this exactly time uh
the work in painting makes the subject
matter of the entire work to be the
process so this is right now kind of the
focal point of uh my
work
um um Now quickly to go into the
technical part so how the Machine Works
more or less uh firstly the name it's a
vrony for electronic drawing apparatus
for vivid image display so obviously we
wanted to call it David and then found
something matching uh we started it 10
years ago and it was really just that
robot uh in the basement it had some
wooden canvas in front of it where we
tried to draw in some way we started in
felt tip we moved uh over to using ink
brushes like Japanese calligraphy
brushes and finally kind of graduated to
um Regular painting with acrylic uh
paint on different kind of canvases and
different kind of
brushes um we're doing it at the
univiversity of constant as we said in
the department of computer Graphics
because it's it's a bit difficult to
find a home for such a project if we did
only robotics painting would be a bit
you know yeah it's a nice application
but not the not the point what we're
getting at and computer Graphics is a
bit of a sweet spots the goals we want
to achieve with this project are many uh
so one is autonomous painting ideally
would like to have to the machine in a
room close the door give it some
instructions then it produces a painting
by itself we can pick it up and it's
done that would be one aspect but also
to provide some kind of tool tool for
the artists so either as an assistant as
mentioned by the Herold so we could have
an artist who just makes a sketch uh and
the robot fills in all the tedious
detail work so the artist can do
something else in the meantime or as
leot is doing uh figure out completely
different use we did not think about um
for this machine because as computer
scientists we think as mentioned in the
video in in terms of controlling this
machine how to do fine detail work and
not necessarily explore all the artistic
possibilities this project is a bit of a
combination between engineering robotics
computer science art and cleaning
because as you also see in the in the
background down there um having these
Robotics and paint which dries all over
the place can get a bit annoying and
cause some problems we also have a
mobile demonstrator for exhibitions uh
not the small yellow one you saw we got
rid of that one we now have a about this
tall um robot by ABB which we sometimes
bring to exhibitions it's still a lot of
work um but maybe in the future uh we we
we can show it off somewhere
nearby um how the entire ma Machine
Works is like this we've got um our
canvas where all of the painting takes
place the robot knows it as its
workpiece and then and by moving along
certain points in the image and having a
brush in its griper it does produce
brush Strokes on the left in the yellow
and green highlighted area you can see
our paint repository which is just a pot
where the um brush is held into in some
interval um picks up some paint uh
discards uh the excess and sometimes you
also clean it and that's just a water
jet where the brush gets um where all
the O paint gets removed and we are
ready to move to over to another paint
the entire process is supervised by the
camera you see in the top left in blue
that's kind of what sets us apart from
other painting robots that might exist
we have a feedback mechanism which can
uh supervise and adapt to some errors
that might
occur um so here in the top left you can
see what the robot in autonomous mode
would start out with it's just a normal
jpeg you would hand it to the robot and
it would compute some kind of paint
recipe and say I want seven colors mixed
in in these ratios from some basic
paints put it in these slots and then it
can start
painting
initially it just places some stroke to
try to get certain areas to match the
input image so the Blue Sky it will
deposit a lot of blue Strokes the yellow
um Sunrise area it will take its yellow
paint and um put it over there the
initial results of course are not very
good because we're trying to get ahead
quickly we're using a big brush and but
after we deposition some initial paint
we uh create the difference image which
you can see in the top right um in red
the areas the the robot considers it
more or less complete so it does it
can't do much more to improve this area
in terms of getting closer to the input
image but the green areas there it can
expect some more progress so for example
the sun uh the yellow area is not bright
enough so it's marked in green while the
beach in the front um it's darkish
enough for this level of um detail so it
just um skips that area for now until it
maybe moves on to a smaller brush based
on that we compute um new Strokes which
are a bit difficult to see but this is
our live debug output which we can see
while it's
painting um which are then stored and
executed in the painting process and
after repeating this many many times
like over one day to sometimes even
three days we get a result like that on
the left which is kind of close to the
input image and we can definitely say we
have reproduced that image in in some
way our early attempts using only the
ink brush look like this um here already
we see some mixing like long strokes for
the tree and short Strokes for the grass
below it this of course was done by
masking the robot doesn't really know
what it's painting it just um follows
our orders in this
case um and after as as I said after the
inks we moved on to acrylic and these
are kind of our Flagship examples the
best results we've got so far um uh
building in the harbor of constants and
a car in the desert and you can see the
um you can see the details which the
robot has managed to achieve using its
feedback mechanism you can also see some
problems we still have like the antenna
is a really small detail and our current
mechanism it's not perfect um there are
many issues in which it bump bumps into
and I also noticed it's kind of an art
to figure out what kind of input is good
for the robot so which whether has it
does it have a chance to produce a good
result because we do have a bunch of
stuff we would um we don't show we kind
of got rid of um speaking of uh stuff we
got rid
of um okay so there was a kind of as a
side effect happening while the robot is
uh operating in automatic modus and
beside the main painting it's actually
creating there is an extra paper and on
this paper the robot is been programmed
just to sweep extra paint that was being
DED uh first the brush was going taking
paint it's taking way too much so it's
going into like the small paper to make
those side um kind of a small brush
strokes and I found this actually paper
to be much more interesting than the
painting itself that was being produced
uh by the system as someone who's coming
from a different kind of a background of
paintly background I always looked at
those painting thinking why would you
like to do this there was this interest
of there is a technical uh challenge but
there is as well the reason of why we do
what we do and I am coming from a like I
actually got educate in in painting
department here in Li it's a very
classical uh Academy for painting in
traditional European
man and when I graduate I remember like
standing there and looking around
thinking hm it's all good but where I am
coming into this entire discourse I'm
coming originally from the Middle East I
have a different kind of a background
cultural background that make you think
why do we adapt different kind of
certain of Aesthetics and how we walk
around it and one of the major question
that I came to understand in my
exploration is that what for me will
make and painting right now the most
interesting is looking into different
structures that shows logic in them
rather to be at the current stage more
interesting than into going into
narrative painting namely things with um
uh let's say semantic uh evaluation
which is I think for the computer part
as well very interesting if the machine
how can you make the machine aware of uh
breaking the painting into to narrative
elements this is a sky this is a house
this is a bird how they can actually be
uh taken apart together so looking at
those works I start actually taking
those pages and I interact with them I
addit some more unconscious kind of a
brast Strokes to go together or like
more gestural one to go on this like
very structured one coming from the
algorithm of the system at the same time
the machine uh the paint is is dripping
there is some kind of irregularity so
it's always about this small um relation
between
things that have 100% organization and
things that are a bit less so if I would
kind of try to think of a a metaphor of
it it's like taking the painting realm
and to track it as a as a rubber um that
you you kind of pull as much as you can
till the point that you know if I will
just stop pulling it right now it have
the strongest like gravitation to to
each other back or it can actually snap
and rip and this is the point that I'm
kind of investigating
with the help of working uh with the
computer graphic department so this is
just kind of implementation that I start
to understand that to actually before
you even go and start talking about
painting one need to talk about the
first thing that what painting is about
and this is about brush Strokes it's
about lines and this is the first thing
of recording of movement and we have
today in or not today for many years
already uh the use of
um a computational aesthetic and Power
art especially in music however there is
something about the sound that it's been
done and it's gone and painting allows
you to actually record every movement to
record every action and to go above it
and to manipulate on it and I found
these elements to be specifically
interesting and that was the base of the
creation of some of my earliest work
I've done in 2016 in collaboration with
Thomas lenm at the time at the
University of constant um where I took
different kind of um parameters that
goes very basic mathematical structures
and to see see how one can build
different kind of um all over
composition a lot of these Works
actually be done on the gal psychology
uh uh writings and I was just trying to
take those ideas and translate them into
basic mathematical equation and execute
them into painting the other uh level
that I start to investigate is the
relation of a single brush stroke that
you could have seen in the ones before
in relation to actually how many BR do
we need to actually create a form and
how that is being put in a compositional
area so it's a very technical questions
but at the same time those structures
are the one that create and build our
perceptions you have when an element
when the individual Strokes become a
spot of pain so when the the one become
a group and how do we like we understand
when we get to the point that we
understand how we make the machine to
understand those situation and when to
use different strategies when making
painting back to be integrated in the
visual feedback more aut like autonomous
painting
methods uh another important part that
uh we worked on is how we can make The
Strokes to look more organic so if you
look in the right low you can see that
uh The Strokes are very straight they're
very they look more mechanically done so
it how can we introduce things that just
a small deviation of the The Strokes or
like the brush not to go like 100% like
0% but a bit like this and a bit like
rotation and I'm saying we want to make
a spot of paint here but don't make it
directly some things to go a bit out and
bit in so all those like small details
is something that for me were very
important uh in creating something that
we as as people can relate to and feel
this is actually a work of art and not
something mechanical and and distance
from us is by increasing the fields of
of organic representation so this is
like one of the work play around that
I've done with Thomas in the beginning
of my um
residency uh showing the work on the
right is done 100% by the visual
feedback while while the second was
being done according to breaking down uh
the entire concept of the painting into
masks and using only three colors so
both of them are being done with the
machine just using different kind of
strategies extracted from this picture
of this lovely guys that he does not
even know that he's being shown here
right now I I have no idea who he is
actually lost to time yeah
[Music]
um and that H developed to the work that
we have kind of funny idea let's make of
course uh what is like when you
acknowledge a painter or like a to be as
an entity then this is a question one
can take as well into the art world uh
discourse the commercial art World
discourse but with the classic
self-portrait uh of the painter and this
is the kind of the preparation phases of
how we done it and that was going again
into the strategy of actually breaking
down the painting into uh masks and
giving different strategies of how we do
those paintings so we actually base this
entire work on a photograph that then
I worked out a bit with [ __ ] and using
the um custom made program that Thomas
built for me I created a different kind
of masking and strategies of brush
Strokes to create the painting that this
is this the uh David um selfportrait
that we find to be very hilarious even
uh um signed his
name and going back like after I've done
all of those work I still had to this
point of
what makes a painting more interesting
why we still interested in it why in
this time that we overrun and overload
with with visual imagery painting is
still important people still going to
learn painting we still do
that and I'm always like tricking
between the elements of making and the
elements of OB of the perception of how
we see it um and that was like a kind of
um this work was built on the idea of
um uh like how would I say like the
action of making a structure of just a
stroke it's very doodle looking
structure that was done by the computer
and something that is still complex
enough for human being not to able to
repeat only a machine can do it but the
deviation of the changes and the
variation of the work is actually being
uh consumed or or created through the
use of paint so how often uh the robot
takes a paint more or less and then just
make a feel oh this it feels like it was
being painted quicker but it was
actually a bit drier the the brush so
how one can make those like small
manipulation and tricks for uh the
creation of painting only in the domain
of a brush
stroke speaking of brush Strokes um what
we have currently as you can see we are
kind of painting in always similar style
we have got a brush we apply it in the
at the same pressure level to a canvas
then we get some paint transfer and
that's how we create our images um
because it's very difficult for robots
to actually handle a brush we don't have
a tool center point which everything re
revolves around in in in these machines
usually you have a welding gun you know
exactly to the millimeter here the
effect of my tool will be
localized um but brushes they deform not
transform that shouldn't shouldn't
happen they get dirty it's really
difficult to simulate a brush because of
these many hairs uh they vary their
properties when they get wet and so on
um and the provided software we have for
handling Tools in robots only works with
these hard uh hard tools basically not
soft tools like
brushes um and that's a problem for our
painting and also what we can provide to
the artists as a control element in here
in the painting process because for
example it's difficult to define a
certain Edge in a painting the painting
process will draw over it many times and
since the brush deviates a bit every
time um the corner will just blur out
hence we can't do many details and we
don't really know the side effects a
movement generates so if you take your
brush move it up and then perform a
sharp corner it will slide around on the
on the canvas a bit uh and produce a
certain effect we we can't control for
that yet so our solution for a stop Gap
solution in the first iteration is to
actually measure measure what our tools
can do in terms of width and lagging
behind so if you look at a brush like
this um we we think we have a nice
defined tool center point But as soon as
you apply it obviously it blurs out like
that it deforms it might stay like that
and as a funny side note if you overdo
it with the robot and have some sign
errors you deform it
permanently um so our solution to this
problem
is doing this um we have some automatic
procedure I'm not going to go into too
much detail but basically the robot can
paint uh Strokes at nor pressure levels
which we limits to avoid what I showed
you previously and then we can use our
feedback mechanism to measure how broad
a certain stroke gets at which pressure
pressure level and using this we can
accurately
um uh reproduce
Strokes for example in this case um we
can account for width we can also uh if
a human Paints the stroke you see on
your left with the red Center Line we
can record it using our feedback
mechanism and paint some first attempt
like like we do normally uh with the
yellow Center Line then we try to figure
out what went wrong when painting this
where might be lagging behind where we
not applying too much enough pressure
and so on WE compute some correction and
can perform the stroke again this
happens over several iterations and in
the end we can see a quite nice end
result but here you see there's already
big Improvement between our initial
attempt and what we can actually produce
and by storing these Strokes um we can
kind of build up a database of knowledge
over time which then artists can use to
tell the robot hey put stroke number 25
in this exactly location thus enabling
much finer grain control and kind of um
as a party trick by just reproducing
single strokes and writing in a funny
way we can also make the robot just
repeat your writing a bit like a
medieval monk if you so will um yeah and
single Strokes um so as you can see like
our work has changed the material before
the work was been done Lots with acrylic
painting means that you can work in
layering
and in order to really um control and to
master the way of we doing the brush
strs we decided to move back into
working more with ink and and paper
where we can actually have the ability
to see every stroke into its details and
to understand how we can control it
better how can we pursue better what we
want to achieve uh this is just an
example to see the entire kind of spread
of a stroke um using different kind of
um of points the say okay we have a just
the time in the the second one just the
time how long it just standing so just
to see how the material is reacting so
we are talking here about not only uh
the computer itself and the stroke but
actually the material itself the the
paint is reacting if the brush is a bit
too wet it's actually expand more into
the paper uh a bit less or a bit more
like the gradients of the color is
changing according to how often it's
actually dip into paint so in this
situation it was uh dipped every 80
Strokes in paint but every 10 Strokes in
water so the water is well slowly
diluted and
um and the paint got as well diluted and
there was kind of situation of gradients
that happened through those kind of a
system that goes to us Indiana's
viewers and together uh for this
work U Marvin had here had build this
software for me that generated particles
and those particles are actually been
created uh due to Newton's law of
gravity um I just decided to use this as
a general law of how we can spread
structure how things are operating in
space if I'm looking at the painting as
a as a space of of creation like in the
world and we can scale it into different
situation and different sizes take only
elements from it um and I use actually
this work to create uh an entire
installation build from many elements
and to ex execute it different parts so
again this entire work was being done
using and the rice paper here just to
show you again the differences of how
just using a different brush make the
outcome to be differently just totally
different although it's the same section
it's the same information that was being
given into the robot for
painting uh the fact that the robot is
not aware of the material is working
with a lot of situation happens where
the material is getting this like really
deformed what probably as a trained
artist I would think it's maybe wrong or
maybe I should not do it or I will try
to avoid it and
the robot is not aware of those
situation but it's actually allow us a
new way to handle with materials to to
see oh it's actually interesting uh
situation to have those holes happening
in the papers like what happened it's
too too much uh going over the
line so those work were being created I
think a 20 something rice paper workor
that I painted taken from this uh um
particle generator and been hanged in a
room in a inative way it's given us the
point again to observe painting not as
one big thing that you have to look s
onent but actually forcing the viewer to
walk by it and to stop and to look about
every little element and to understand
and discover the process of making it so
making the process much more
visible uh this work again was been
later develop into how one can exhibit
the work so to go outside of the
painting from just think this is flat on
the wall and to take it out into the
space uh so therefore there was a
decision of uh framing those Works in
double Sid uh using um a hanging frames
and you can actually look at and see
them from the side it was being painted
the time it was being uh just absorbed
and creating a special area where one
can work uh and experience the process
of the creation and not only the Finish
object as a
painting so there kind of uh where we're
at today um but the project is of course
going to continue and be developed
further over time one major area from my
side I want to work on is painting
technique so right now we've got one
style we can do well um but of course it
would be better if we could more emulate
what the human actually does a human
does not stand in front of his canvas
and play stroke after stroke in random
places um we would like to make the
robot scribble on the surface mix some
Strokes deliberately Maybe perform abs
and Vary that style within one painting
maybe as is appropriate um of course
getting a machine to actually do that
requires a lot of um input of certain
movements and it's a bit
tricky also I mentioned the database of
movements we'd like to build so we know
okay this movement will give us this
exact result so we have a much more
predictable um process and something you
can pick and choose from and maybe even
tell the robot okay only focus on this
type of stroke instead of this so you
can in and guide its style in a much
more concrete way than right now where
you can only very stroke length and
stuff like that also our system design
you saw a bit um of the development we
started out using a plastic bucket with
some Grid in it to clean the clean the
brush uh we're moving on prototyping as
we go on and trying to develop better
Hardware so the robot can actually have
clean tools uh save tool changes and so
on also openness is a bit is a problem
at the moment um it's a bit difficult to
get other code into our project or to
just get get strokes from somebody else
he would like to have executed on the
machine so we're reworking uh the
software architecture to make to make it
possible for other collaborations
because it turns out we've got
competition in some way in a good way in
toine I think there's also a a painting
robot being developed at the University
we are cooperating with shenzen
University who are kind kind of
reproducing our robots and then we have
two of these systems to work with in
different places also because I have to
mention it machine learning um we need
to figure out if it makes sense to
actually use machine learning in our
approach if there is some Easy Way um to
to do it um is it possible to maybe
simulate some creative aspects you know
these Google deep dream thingies could
we take that technology get out some
strokes and have uh have it output on
our machine and the interesting side
note who actually gets copyright if we
allowed to do the M if we allow the
machine to do more and more by itself so
at some point we had a law professor in
our lab and um he asked how much are we
still doing in this process because once
the machine starts doing enough and
reaches What in in German law is called
Chop I think uh it will get difficult to
attribute to who actually owns the
painting which is being created and some
f future issues for
leot um Okay so the work that we're
being done we're still working on
stopped for me right now in the moment
of how can I use this tool to extend
myself it means how can I go beyond the
limitation of myself as a human being
what I can do first of all for my own
perception because when I'm standing in
front of a can a canvas or a painting
surface I'm looking from one place my
breast strok ability is only limitated
to my sides of my arms so I would like
to integrate more into those kind of
elements to see how can we use this tool
to create maybe a larger scale works at
the same time uh idea of how one can
translate data taken from different
elements into the painting so the
painting will become an abstract
representation of maybe social structure
if we take it from movement data that
I'm right now collaborating with a Kaza
Paganini in in Geneva taking data from
behavioral uh movement information and
using individual human beings actually
to be translated into BR Strokes
um further it's the idea of how the
machine could become more and more
organic in its Behavior so it's not only
the way it looks but the way it works
and operate so right now it works very
deterministic ways like if you give it a
JPEG it works from it according to uh
the parameter it's being given or it's
giv through a deterministic way of um an
algorithm and being given and just being
filling it up I would be very
interesting to get to the point where
the machine will integrate what
considered to to be um a mistake in the
painting process something that we
didn't expect it to happen we didn't
want it to happen but it's there now and
it's changed the entire world situation
so how can we integrate and react to
each other so the coners the
conversation that happening between me
and the Machine is actually there for
opening a new realm for creativity
because there is something almost
objective that comes from this kind of
an input that makes me question myself
of my decision- making process and my
habits of visual information
um yeah that's kind of what we have to
say so far uh if you want to figure out
more about the project we've got a
website from the University if you want
to see more of Leah's artwork she has
got a website as well and what we
presented today more or less the stroke
thingies you can see in our latest
publication at mdpi self-improving
robotic brush stroke replication a
professional would have added a link
here but um we added that last minute
I'm sorry but you can you can find it
easily I have to use the term you can
easily Google it sorry um yeah that's
our talk so far if you've got any
questions I guess now is the time ah
super
great I'm actually wondering how you
make a robot really Shi though at a
certain moment you have something like
hey come on let me get into this because
isn't it because that kind of
communication would be interesting with
an assistant you just kick it and he or
she goes away but e David it can kill
you if you actually disturb it work the
thing has a lot of power that's
experience we have in our hacker space
as well we have a diamond cutting thing
and there the thing drops sometimes
whatever we have lots of questions I
assume really start number two please
hey thanks for the talk uh if I
understood correctly it's not a printer
because you get more because you get
visual feedback yes that okay so I'm
wondering um what how does it actually
can you scale Beyond just visual
feedback I mean there are a lot of
different ways uh but Is it feasible and
that what would be the consequences
because it could be another social
feedback by for example us or the public
or the audience judging the result I
mean there are so many ways so how is it
how is it feasible practically speaking
what are the consequences so the the
question is can we introduce some other
feedback other than just V visual
information uh yeah higher level more
conceptual yes you could so in theory we
we don't have it at the moment obviously
but if we had a way for somebody to kind
of give the robot an overlay and say
this corner isn't good it's kind of
deviates from my Optimum by a certain
value then we we would also need some
function to determine how it should
improve that and then we could also
include other kinds of feedback of any
kind basically the trick is always to
find a function which describes how big
the error is for the feedback and what
can be done to fix it in cases of just
black or white or color color spaces
it's kind of easy if it if it's too gray
you paint more white color over it but
in here it gets difficult but it might
be doable or do you have any views on
that well I think it's there is a lot of
challenges to use of the visual feedback
because there is the question always of
perception when the visual feedback that
what is actually getting right now it's
from a photograph where everything is
being scaled equally and there is
certain kind of parameters it's checking
if what Marvin just said about the
darkness the level of uh areas and so on
and it still does not have the tuning of
understanding for example semantic uh uh
situation to say oh here I painted the
the tree as part of the the sky and it's
supposed to be two different things this
is the thing the interesting part for us
as human being to interact and slowly
teach the machine and see what it's
doing and correct it and hope hopefully
like L the machine will remember through
its doing um to get to a more defined or
abilities of feedbacks that are probably
much more developed from what we have
right now I would love to see your live
demonstration here uh number one please
shoot thank you for this very
inspirational talk I actually got
several questions but I will limit
myself to two if it's possible one one
one so first question is um could you
please elaborate a little bit bit more
on this aspect of cleaning since you
said it's a combination of art Robotics
and
cleaning and then question number two
would be about yeah what what would for
each of what was for each of you like
the most surprising in this project and
cooporation yeah okay so the first
question was what what's all this about
cleaning I mentioned basic I mention it
so much because it's my major headache
in this automation process right now
because over time when the robot paints
a lot with that brush some stuff dries
some stuff just it sticks in the brush
and it's really difficult to get out
then you contam contam contaminate your
other paints and everything tends
towards some gray brownish stickiness I
think is what it's the differences of of
how computer science are used to work in
certain kind of a lab and suddenly they
have an atali happening there yes so
when I was there for me it was like
actually it's a clean it's a fairly
clean Studio here so I think it's it's a
lot of matter of of habits but of course
when you work with u with machinery and
you work in in a scientifical
environment you need to measure
everything so keeping a clean
environment is important to actually
follow about the results and that I
think one of the reason why it's such a
big headache
for wouldn't it be better to come over
to the aier then with your robot since
that's the atmosphere and environment
two different cities it's too far away
from each other yeah we try that one
this one and also another problem is if
you see the table with the colors on the
left there's also a magazine where the
robot can swap out brushes and once too
much paint had accumulated in there um
the tool did not properly enter the slot
uh it was misaligned the robot then
almost threw over the table when trying
to switch tools yeah so that it's also
just a mechanical problem after some
time but the floor doesn't look like an
natal here no I think yeah when you're
there you see a lot
more okay and the next question was
should we do the second one or what was
the most surprising surprising part of I
surprising I'll be quick when I entered
the project I was like do we have a
robot yes can I work with it yes do you
want to know what we do no um and then I
figured out oh this art part exists I
had no no connection with it at all and
that was kind of surprising for me to
bit stumble into this uh World which
which also is included in the project we
can imagine a little bit number two
please yes oh yeah the surprise yeah um
I I think from my side it's fairly
similar to uh from maybe my Marvin Parts
because when I came there I thought okay
it's going to be we coming from very
different world how can we discuss very
similar issues and it was a pleasant
surprise to discover that we have
similar questions and we look into the
world into very similar ways but the
tools that we are doing it the
perspective that we're doing is very
different so the attempt of trying to
communicate those to was for me the most
interesting part and just looking into
structure to understand how can I use
computer language structural language to
actually build painting and how similar
and close to each other those two
are organizing chaos or
chaing organizations something like that
entropy yeah please sir hi can can the
robot make use of some tactile feedback
for example poal sensor in the
brush um we don't have it and it's it's
very difficult because we the forces
that should be occurring are very small
and might also change because of stuff
accumulating um and also as soon as we
introduce a wire into a brush for
example to measure something that will
change the outcome um so no but we are
thinking about maybe having some live
supervision of the brush with with the
camera um but then you get the problem
of a green brush on a green background
and trying to separate that it's it's
difficult so we're still painting more
or less uh blind in that regard if you
have a solution come up and hand it over
number two one number one
please uh as an artist experimentation
is very important to try experiment
experimentation is very important to dry
out new things to you know do things
that you maybe didn't plan to do and so
on so on on what happens if e David Gets
a brush that it doesn't know if there's
a tool in the in the tool box that it
picks up and then just applies it to the
canvas I actually done that with it
several times and it just makes the it
just make it keep painting if especially
when you work with a visual feedback
because things happen that you didn't
expect it to happen so the machine think
it's wrong I will just do it again and
it could keep probably paint like this
for eternity because it doesn't have
this like you know experimentation
something that you can tell to a human
because then I experiment something new
and I understand something from the
process and I'm learning from it the
machine is still not in the situation
that it's learn it has a goal it has its
tools and it's know how can I use those
tools to achieve my goal it doesn't
understand swearing or something like
that no no but sometimes I do swear at
him or love to it depends it helps to my
psychology we have a couple of more
minutes uh it's very interesting what
you say because obviously it's dangerous
because the machine does not have much
sensory feedback but if we do it in some
controlled way where it knows limits so
it must be constrained in that way we
could definitely Implement methods where
it just paints randomly records it and
then somehow uses it in some further
experiment
yeah number two shoot shoot yeah uh
thank you very much um I have uh two
more of art historical questions um
first uh the notion of the painter right
the notion of the artist painter is um a
otion that is like marked with a lot of
um markers of gender for example like
the men that paints but also of
virtuosity of authorship and things like
this have you think about like um
dealing with robots with other media
that isn't marked this way or maybe with
painting but differently or how can one
change this notion that kind of gets
repeated and um also I was
[Music]
wondering
um
moment yes oh yeah yeah yeah yeah I was
wondering about um uh people like John
tingly like uh people that would were
dealing with machines already I don't
know in the 50s or something but they uh
understand themselves as as sculptures
too like they were producing a machine
that was mechan meal and there was
painting like a painting machine and of
course what you're doing is differently
since it's a robot but how like if you
relate to that
tradition thank you
um so the for the first uh question uh I
am a painter and it's important for me
to State it although for according to my
Academia where I studyed they consider
it not to be a painting anymore because
it's not exactly done by the authorship
of a human being although I'm working
with uh paint and brushes I was being
sent directly into a video audio
department when I wanted to produce to
show this project I was like I there is
no audio or video in the work why you
send me there um so when it comes to
using different media for example now
the project I was doing in Italy was
working with um using digital
representation and different levels of
visualization that I found interesting
but to be honest for me the interaction
with real materials is the most
important part and using uh the machine
right now is an extension of how one can
use technology for its own creativity so
how we we change like so I trying to see
it in the relation of art history as an
opportunity to establish a new aesthetic
and not to go into the discussion of
authorship although it needs to be there
and it has to be there because of course
when the camera was introduced
photography introduced like in the
beginning considered not to be even art
and it was very low valued because the
machine done it not the person today the
discuss is not there anymore so I think
there is this Freedom that I have right
now in this domain because I don't have
the AL to sit and look and upon and say
I have to imitate those like old
European men and the way they've been
painted because it's new so I can do
whatever I want and no one can tell me
it's right or wrong and this is a great
privilege um for the second question
about the way of presentation I will
still go back to say I am a painter and
for that I'm interesting in the image
itself and I think there is the fism of
the machine especially today when you
see the robots and people don't even
care what it's doing they see a robot
moving it's like waa it's so cool and
it's nice and I think you can do a lot
of things in U levels of um interaction
works and
um but I am really not into this like
it's I I'm not in I'm not a performative
artist I'm not interested if uh making a
kinetic art I'm more into really just
how can you represent the movement in
the steel in the Ste image and how that
could in a way fight back against the
overload of visual imager we have today
because we maybe look at the painting
for 3 seconds we get it we go on like as
a visual artist today I'm being required
to know who are my contemporary fellows
I cannot be at the same time in Hong
Kong and in New York and in Paris so I
check them out in my shitty phone and I
see this imag I like oh I get what it's
done there and I'm continuing so it's a
different way of experiencing
information and I want to use this
machine as a way as a tool to
communicate back what the information
coming from the machine from the
computer that we are all uh using right
now and reconnect and
of course there you would need to define
something like a cost function or a loss
function which describes how well the
system is progressing and I'm wondering
both artistically and scientifically how
would you go about what are you thinking
about defining such cost or loss or
quality
functions yeah um that's what exactly
what I'm working on right now and what's
giving me nightmares because it is very
very difficult you try to reduce the
problem and then you already impose
extreme limits on the machine so what
would maybe what would maybe be a good
first step because at the moment we do
no learning at all in the current setup
um would be reinforcement learning and
just have the machine approximate some
kind of template and maybe afterwards we
could say certain class of stroke is
aesthetic in some way but it it gets
very vague and difficult so we'll we'll
figure that out in the future I hope qu
but also I guess artistically it's a
very interesting question right have you
talked about you concerned right how you
define those questions this is a always
a what is wrong and what is right in the
Arts what so
it's it's an issue if if we find
something I think it would be troubling
if we could say yeah this old painting
is 85%
aesthetic yeah but I think there is
today a huge amount of studies that I
found like in our science that I found a
bit scary to be honest they trying to
quantify those those elements and one
can look at as an absolute because it's
all it depends on the place and the time
and the history that you're coming from
and this is a huge challenge that we are
facing today as we all live in this like
Global society and everything is
flatting out and everything that's been
done and being taught in art cies across
the Globes are kind of similar so I
found it to be very dangerous and this
is why for me this working with a
machine is not about giving it the
autonomy but using it as a tool to maybe
even like learn its individual user so
maybe we can create several kind of
users and each uh user learn the machine
will learn the habits of its user and
therefore would continue into a relative
absolute uh values I would say I really
have to apologize because we have to
close this session I'm looking forward
to these new chapters of artistic uh
behavior in humanity I suggest to
continue these uh speeches afterwards
somewhere here in a car in a bar or yeah
you be will be available here I would
love to thank uh lead Marvin E David of
course and Professor Doan that's the
name isn't it please thank
[Applause]
[Music]
you
[Music]
oh