Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/111 Thanks!
To talk and start and yes,
it's about metadata or
against metadata, I think everybody
knows about the metadata by.
So I think I don't have to tell you very
much about the talk.
And so speaking is
Robert Oxhorn.
Yes.
Thank you.
Thanks.
This is what a digital video looks like
without metadata, you are
seeing and hearing a stream of digital
bytes, unlike
a frame of film, which is two
dimensional, no shape in
here is within computer memory.
When we save data to disk, we
must store metadata that allows for
its familiar representation.
In this case, the width and height of
the video are unknown.
And what you are looking at is a user
interface I have developed
for rediscovering the width in the height
of a video.
If I move the mouse to the upper left
hand corner of the screen,
the video plays as if it's two by two.
We can set the width and
the height.
We'll come back to this in a moment.
If we consider that
to be that, which I call signal an
example of data without metadata,
then what I want to show you next noise
is even a lack of data.
The objective here is to cancel out
noise.
I can move the mouse to position
a layer of antennas which is
identical to noise,
but I have no hints as to where they will
meet
if I zoom in.
A little more hope
and see.
Is there sound, by the way,
or is that something that stopped?
Here I start to see the patterns emerging
as the anti noise meets noise.
And then there's nothing
there's nothing that can be found in
noise, the only pattern is itself.
You might have noticed some visual
similarity at first between noise and
signal.
Without metadata, it can be hard to tell
the two apart.
I'm going to come back to this for a
second and try to decode it
sometimes a little difficult.
And moving the mouse around to safety
with an.
The video that I'm trying to decode
was released by WikiLeaks
under the name Collateral Murder.
I developed this interface between
2010 and 2011 in conversation with
my friend's death, inspired by a remark
Julian Assange made that
his organization had decrypted the video
prior to its release.
We wanted to give others the opportunity
to decrypt the video, to actively
take part in overcoming
the formidable hurdles we face to bear
witness.
From what I can tell, the technical
reality of Assange is playing this
controversial.
From the test to the
trial of Chelsea Manning,
there's some indication that the soldier
had access to an unencrypted
version of this video.
Still, I think there's something to what
Assange says, said, because
even if the organization didn't perform
magic crypto wizardry to
bring the video to our attention, if
he wasn't using this interface or
anything remotely like it.
He did decrypt it in a way,
because the release of the video,
starting with its title as Collateral
Murder, was not merely a release of
raw material, but also metadata.
The video was not decrypted to find it.
Machine readable metadata, the
crypto protocol, privity, audio video
codec with height and frame rate.
But it was decrypted to make human
readable claims about the video.
I think I'm about to give up on trying to
solve it.
The title of this talk is Against
Metadata, submitted in the early midst
of dragnet surveillance disclosures,
but perhaps after a more sober
consideration, it should have been
called beyond metadata or towards the
new metadata or some such.
The summary of Snowden as apparently
here is known evoked
in me a sort of double sadness that all
this metadata had leaked and
perversely, that the dragnet ran so
shallow.
Metadata is too banal and frankly
necessary to antagonize.
But a plea stop designing
software that assumes the prior existence
of metadata.
I would like to split metadata into three
categories.
First, there is machine metadata
designed for the computer to interpret
out of our sight, such as the width and
height of a video stream, its frame rate
codecs, etc..
Then we have metadata bases full of
relational columns that, more so
than the original, are suited to rigid
database systems and which are then used
to stand in place of the original data,
such as a book, cover titles
and names of all sorts icons, timestamps,
GPS coordinates and so on.
Finally, the last sort of metadata I wish
to consider is interpretive metadata,
which is a layer of data describing what
we make of the data.
Compression is the basis and enabling
force of networked media, and as
with all machine metadata, it is deemed
successful when it is invisible.
This is an image which is represented
digitally as an array of pixel intensity
values.
I can mix wires and turn up the volume
so you can hear these values
played back.
Compression inherently involves a
determination of what information is most
important and a purge of the rest.
For example, compress images tend to
assume a basic continuity that
pixels are similar to their neighbors
and therefore operate on spatial
frequencies of pixels.
So while we perceive this image as a two
degree grid, the computer processes
it a little bit more like this.
And I can take these frequencies and one
at a time start removing them
like you can hear, I'm
removing the pixel, the frequencies that
are the least salient
and I can flip back to the image and we
see that it's more or less unchanged.
This is with 50 percent of that frequency
is removed, which you may be able to
hear.
I can keep going.
About seventy five percent,
nine days, and
finally all of the frequencies
are removed, I can add them back
one at a time.
And we can see the images start to become
recognizable again,
what would image compression looked like
if it was oriented towards humans
and not machines?
These are a series of portraits,
and I've
given myself a parameter over each image,
so I may compress it
by horribly abusing the math
from some carving
popularized by a paper on contemporary
imagery resizing, it was designed
to invisibly change the aspect
ratio of images for different
sizes of displays.
I hope that by dialing up all of these
images at once.
I can get a few laughs out of the
audience.
I see the ideal of metadata as
encompassing the meaning of that data
and compression as a proposal for that
meaning, if you laughed at these
images, perhaps it is because you got
more than you expected for them, that as
a caricature, the compression functioned
as an exaggeration of distinctive
characteristics.
In nineteen fifty, the pioneer of
information theory, Claude Shannon,
wrote, The meaning of a message is
generally irrelevant.
When I read that at first I took issue
with it. Shannon defines entropy
or information relative to the average
or expected message.
Information is surprise, Shannon
writes, and the encoding of a normal
baseline into the superstructure of file
formats and media codecs seemed
a reactionary bias towards mediocrity.
From telegrammed to t nine codes.
A machine may not be able to comprehend
meaning, but it can be programed to let
some pass through a slit.
And this is another side to Shannon's
statement that perhaps I missed the first
time around.
Meaning need not be encapsulated within
a packet for it to be carried through the
medium.
For temporal media, sound and video
codecs have given a greater and greater
instantaneous fidelity, but leave
us with few techniques to skim, seek
and survey, the timeline
is a fixture of digital non-linear
editors, but rarely in our players is
it much more than a line.
This is 60 seconds of video represented
with 12 thumbnails, each spaced five
seconds apart.
This is the same 60 seconds, but now with
twenty four thumbnails, two and a half
seconds apart, left, half,
right half.
And when I keep cropping the images more
and more down to a single
pixel, all of a sudden
form reemerges within
the image. People in cars are visible
again.
Movement is frozen through the column of
pixels I choose I take
in no matter which column that is.
I can look at the next 60 seconds
and see a similar effect
or the next 60 seconds.
Now we see a parabola and
when I uncropped the images, we see a
series of them.
I'm interested in a point about here
where you see the form, you see the icon,
the clock, the buildings, but you also
see how they relate to time and how they
change the part.
And the whole
the timeline can function as
a graphic metadata
to communicate the contents of video
without, in fact, having any idea of
them.
Here's the timelines index, 15
films by Cut
at different phase.
And I can play the video through these
timelines.
Using them as index.
These timelines eventually turned
into a Web based documentary
I did in collaboration with the filmmaker
L7.
Here we have metadata as the
film.
Tags serve as entry points to the
material.
Nobody can say that
it is to blame also, do you
see that one
born
again we have these scandals
usually created more was
on the tags, allow us to pivot
to change how we're seeing something
without it stopping so bad
that I haven't, you
know.
In the middle of the
day, and I'm older than me,
but I am little.
Metadata as authorship is only the
beginning,
for if this is the real time generation,
real must be intended as in reality
TV.
The rise of portable MP three players
gave us the opportunity for our lives to
have soundtrack's and the smartphone,
NSA spyglasses, etc, all
give us thickening metadata layer to
lived experience.
We are making history as we are living
it, which I mean less than a
congratulatory reference to our
significance.
But simply pointing to the accumulation
of documents and data is that we or
someone something must sort through.
Here I am live coding a video editor
to organize a screen, tests of myself,
life coding, a video editor.
It is the classic Burhenn problem of
the map, and the territory for every
attempt at curation is also recorded
and reinvested.
Bernays wrote about the empire's
fanatical attempt to create a map so
detailed it ended up as large as the
space it sought to describe the
map occluding and suffocating the
territory.
But that was before the advent of slippy
maps, before level of detail
distillations.
We had the present moment.
And while a map of the future is hard to
come by, we can control our view of
the past with a fluid zoom control.
Try to work your head around the passage
of time here in
the video metadata, the traps us neither
in the present or the past.
Applied back to the first time lines, I
showed you those from John Smith's
brilliant nineteen seventy six short
film, The Girl Chewing Gum, itself
a biting critique of cinematic authority.
We have an experimental film
I made called Chewing.
Notice how time passes and also
how it doesn't
how characters are endlessly looped into
themselves.
So we can design video codecs that give
rather than remove context,
all of the massive codecs is valuable.
And I hope that some of you who
intuitively understand it better than
myself will work on opening up
codecs to our perception
to show you what I mean in a different
way. Consider the matter of a video
stabilizer.
Put simply, a stabilizer deforms each
frame to the previous, and when
engineers design stabilizers,
they zoom in to hide the edges that are
irregular.
But what if instead of zooming in, we can
zoom out?
Now we have context,
we have what is no longer seen, I call
this technique periphery in admiration of
our perceptual system, which gives us
so much out of sight context,
I hope I've suggested a few ways the
machine metadata can be designed to
reveal rather than obscure and
have the inevitable metadata as we form
can empower our curation and
comprehension rather than functioning
as advertising and entertainment.
The most practical software that
I've been designing for this
is called interface.
I have very limited time, but I show
you very quickly,
um, a
little screenshot, a little screen of
a project I did in collaboration with a
conference suitably enough for my
metastasize about video
that perhaps instead of just
understanding that as reaction,
there is the entire conference which
is visible through a zoomed assumable
timelines.
And then there's a series of footnotes
that provide an alternate mode of
navigation. He talks about
it back in his
dorm room in Georgia while.
There's a theory, if you want to change
the world, that you have to do it by
helping people share cat videos.
I don't quite know about that, but I'll
show you some thing a little bit less
serious fed into this sort of machine.
See how it does
and how
maybe we should take questions as you
look at this snorkeling video had some
other things, but I have absolutely no
time anyone.
If there are no questions, you have five
minutes.
This is called this
and the software is codas
on README not button OMG.
There have been a quite a few iterations,
but now the latest is interlaced for
Alpha Grid.
Be patient.
That's one question.
I guess I'm taking your challenge
of curation quite literally.
I'm an archivist with a
collection of archival films
and something like this.
On the one hand, which provides
an overview of the entire contents
of a film, would be incredibly
useful as a
tool, as a finding, a way to our
collections. But on the other hand,
if it is video,
there's we have to design.
What we put out on the Web
is a finding aid for the absolute
lowest common denominator of
technology of users worldwide.
Is there a tension there?
And how would somebody like
how would record keepers like me
sort of smooth over that tension
in terms of lowest common denominator?
You're looking at a Web browser
and pushing the limits of the browser a
little bit, but it's HTML5.
It's all open source.
There's no lock in and there's
no barriers to free implementations
that would provide access.
Usability is still something I'm working
on. On the other end, yeah,
I guess I'm thinking more of bandwidth
because something that's sort of a static
series of images
takes up a lot less space,
really, than than sort of here is
on the other hand, having a visual index
lets you watch the part of the archive.
You're interested in downloading the
entire file exactly as
more bandwidth.
I did a lot of optimization to try
to make this like vaguely palatable
over a network connection.
But I mean, these are tiny
little videos.
They're
just indexes.
It's like a search index.
But for video.
And I don't know if I've answered any of
you, please maybe ask
the most important again.
Yeah, actually I can't
remember what it is.
I'm going to have to
basically think about it.
But then we have another question out
there. If you have a question, can you go
to the microphone and
my phone number one.
I think that was really great.
Unfortunately, I don't have the pleasure
of working with video as my main
source of data today, and it's
much more dry a context of numbers and
context.
And I can see some
similarities in what you've done with,
you know, spear cons and spin
Dex's or however you supposed to
pronounce it with sonification and
some visualization.
But I want to have you.
Yes. Well, I'm just going to ask you to
do that. Yeah,
I had where I wanted to show,
but I've tried to apply the same
paradigms to dump's of unsorted
freedom of Information requests and to
sound with spectrograms and provide
ways of not linearly
exploring all sorts of different kinds
of data that doesn't have a clear
archival canonical metadata available
to it.
If there's no other question, I think
it's to say thank you again.