Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/168 Thanks!
Welcome, everyone. Good evening.
Thanks for attending this stock.
So I
don't know if this is true of all, too.
We have another number.
We know we have a centering team, which
is called Buetow, but he couldn't make it
for three this year.
So we basically console arcus
and every bit of developers.
So we work on Wilkins.
Wilkins, are we on the GameCube?
And we like developers of the
dolphin and we are told which is a game
to you when we have data and where
we end up on this.
We use reverse engineering project by
friends from Fay Overfill, which might
have seen on day one.
So basically, Crux opens with
you and made it possible to do this work
so things are to them.
So for those who don't know what, so
you can this so this is a
Wii U gamepad, this is what you actually
connect to the console to
play games. It's
it's codenamed DLC.
So we're probably going to use DLC a lot
in this presentation just because it's a
lot shorter to write and stuff like that.
So just another DLC
is basically a game pass
that is just like, as you can see, still
track your standard
GameCube control finger with
just like two joysticks and a few
buttons. This has actually lots of
features, so
it's a benefit feature.
It's like it's like it's a wireless
device that you don't have to plug it
to where you are, and it's kind of
sending nowadays. But still, it has
an 854 by 480
touch screen, which is resistive.
So it's not like capacitive touchscreen
is just resistive stuff.
It has all your outputs and also
a microphone and two gobies microphones
as a feature camera, which outputs
a 640 by 480 resolution.
It's it's all size like the standard
gamepad stuff. So analog sticks,
19 digital buttons,
motion kind of commotion, stuff,
accelerated gyroscope, compass
and of course, a wireless stuff
which are interesting like NFC
and which is used actually to
use a gamepad as a TV remote like it can
communicate with your TV over
infrared.
That's one big point was it is that it's
actually only juice boxes and into the
EU. So unless Microsoft
and Sony hack six books, one
finance eps for the actually told people,
we're going to die at some point and you
can use it with your PC access to a
Nintendo that never said anything like
that. They just don't plan it.
They're just doing that for it to be open
just like we were before and all of their
other devices.
So it's kind of sad because it's
a really fun device. Actually, kids use
it to actually do a lot of stuff surface
control robots or just play
your games like without
being in front of your screen.
So, yeah, let's figure that
this is like the whole point of this
talk.
So let's open the Xbox.
This is a model both of the video game
pass. As you can see, it's actually quite
complex. Vice Zwane's
chip and this is
in the middle. It's a big one.
It's actually an estimated CPU from
a sea work photonics.
It's called the UIC Sweden UIC,
and it's this is
a first I've actually free CPU's that are
on this game pass.
On the left, you can actually see that
there is a big empty space see you
connect to Wi-Fi chipsets.
It's actually can actually be detached.
It's a Broadcom Wi-Fi
chips that's kind of standard and
also button way to also have an empty
space, which is for the NFC antenna and
AFC control.
This is the other side of the Muslim
world with more interesting stuff
is that you like as a White House is
basically a fascist that.
Well, basically all the clothes and
footwear and stuff for the gamepads.
And this, if you were just as the left of
the flesh is called, it's
written on a TLC dash.
The bow up and the unions are knocking
on each Japan.
You have another indication of what this
ship is.
And we see that on that it's actually the
main CPU that controls
most of the game part.
This is on the other side of the system,
so you must vote not is a gamepad
sons left. You can see the main the
main bundle with the CPU and GPU
of the Wii U.
And at the top right, you have the Java
edge, which is basically
just the same as a DLC
microcontroller on the gamepads, but it
handles communication
between you and the gamepad.
So how does it actually communicates
issue because a back has a wife had, if
that's actually indicates 5G cohorts
for use in those only in your wife,
I said that's very common.
So you can just look
in the line and start sniffing
life and the five, she goes for
coincidence.
But obviously, it's encrypted.
It's not that easy.
So just a side note, this
is actually a big problem with this
project, finding the perfect working
life device is
it can actually do access point mode at
five of five girls channels
is really hard, especially
on windows is basically Molesky
Impossible on six.
I think that some people actually should
get access point of view, but not with
all the features we need for this
project. I'll give exact you only a few
devices are able to do that.
So for example, if you take into
wireless devices, there is
no AP mod support on most cards
that supports 5G goes on across
devices.
You actually can find a lot of these
cards, but they have minor issues, which
are which I'm going to talk about
car in 1991 70s, basically same
HHI and keep it for as.
For some reason, it's very hard to find
devices that use this chipsets.
So is it all the same minor issues
that has no H9 because
it's basically the same, the same chipset
inside, but they also have the same minor
issues.
And then this one
is interesting. The bugging arty to
get under the USB is actually a nice
USB chip, which is not too hard
to find, as no major issues with this
only started working with Linux
3.7, which is fairly recent.
So it wasn't actually working when you
started this project.
And yeah, we might actually be able to
work as well on Broadcom chips, but we
haven't tested that it.
So let's work on decrypting the
communications between the video and
the game, but we can do two things
like first of all, always is actually
encrypted, so maybe WPA or something
like that? And what's the key use for
encryption?
So this was actually fairly easy to find
strong signs of the encryption scheme.
You should just look at the items we have
and there's lots of packets that you save
it. So it's just like
WPA two packets.
It says like this protected bits at the
idea to so that as yet, there
is basically
integrity checked by the encryption
scheme and you have eight bytes
associated with it.
You have another six out incremented for
every packet.
And this much is what WPA
$2.
So it was just forget it was really
actually this was using WPA two.
So we just need to keep now.
And this was actually, as I expected,
actually, no, it was well,
they made it easy for us to like just
having to access.
It's actually have nice features of you
when you want to connect again back to
you for the first time, you
basically you have to pair it with a
console.
And what this phone system does is you
have eggs.
So we use this phrase like four symbols
on the screen and you have two types of
the same four symbols in your gamepad.
And then there's a game band could
connect with you and look at
those.
It's actually like using WPA.
So WPA welfare standards,
that's useful auto configuration of Wi-Fi
devices.
And basically, what it does is use the
UPI because access points you don't know
about and it gives you all the
information it lacks, including SSA,
the encryption
methods, encryption keys and everything
like that.
So the way it is is secure, well secured
by the need to press a button, the access
points or to enter the pins, as shown in
the access points to prove that you have
physical access to a
the access point that provides you with
the iPhone Network and
the specifications.
Kind of interesting because it's it's
feasible to make a lot of sense.
It's extremely complicated for
something that should be simple, like
just check that,
so check that the gate press on the
button and sends them back
data.
So this actual WPA works,
it's actually like eight steps of
protocol.
As you can see, it uses like encryption.
If you have an h mark
with two separate hashing algorithms
like Houston, coded activation
functions and a few of the stuff like
that's ever going to go off of it.
It's interesting thing to see in there is
that basically you only get the config
data to end and that each
step basically checks if
they do like an the back of the press of
this message and the previous message, so
that you can check that the message
messages well and check tampered with.
So obviously, we try to pay a PC
with a redo just using WPA supplicants,
which is like a standard
Vinick's program to the developers.
And it's failed.
So as you can see, I can
this work I've done it felt at home
to editors as the second
stage of this story of the future.
And what's interesting is that it's not
actually that we use and
you get send them a message.
It's actually us not like
not recognizing what what
you sent us. So WPA, so we can send
them a message so we can sometimes
get to you.
So we looked at it a bit more like
is actually figuring it's another
ceasefire tag back
of the previous message was this
message is actually not correct,
like, which would mean
this isn't. The interesting thing is this
is that there is no user data.
We don't depend on the user data for this
age match.
So, so that means that basically
it's encrypted was changed.
So they took WPX, but it's not exactly
the way it's changed stuff, which makes
its failed attempt to.
So I always like
what's actually failing.
Is that the key that's useful.
The it's not the same in the
old keys is derived from
something called decay, decay, and
you can see it uses like the string
constants, wife values and sexual keys
elevation.
So he takes the key from this key
derivation function, and it uses it
to, well, this is what
you just switch back.
So KDKA is derived
from and one and two which
are basically random values and Mac
address.
So, yeah, nothing depends on user
data. Other thing depends on symbols that
we should have to enter.
So everything should just work if there
was nothing that doing a man in the
middle. But it doesn't.
So that's when you actually started
to do some this stuff.
So let's shuffle explain
how we actually works on dumping this
fella.
All right. So when we got the gamepad,
one of the first things we did was just
inspect the board and dump all the
flashes that we saw.
And it turns out that
everything is unencrypted and plaintext
and the faces were really
easy to connect to.
And
we did have to make some changes to flash
ram, but it wasn't very difficult.
And as soon as we did this,
we could start looking through the
firmware that was contained on the chip
and look for the interesting
Wi-Fi stuff.
And I just have a picture of
our respective setups.
And then.
And so before we could actually
look for Wi-Fi code, we had
to determine the
file system that Nintendo was using on
that ship.
And it's just some custom thing that they
threw together with a really simple
header, and it contains
two different banks. So they support
safe flashing so they can fashion your
version and switch to it
atomically and
within the blood or within
a version
of the firmware. There's multiple blobs
which are obviously
Wi-Fi, and you and I, among some others,
which are like picture resources.
But these are the main interesting code
ones.
After inspecting these blobs,
we found out that the obviously
is for the main DRC
chip, which is arm
9:26 egis
and the Wi-Fi babas for
the Broadcom dongle, which is
ARM Cortex-A53 code and
the year my board was for the estimate,
USC controller.
So we knew from before
we knew that this string was involved in
computing the chaotic
and simple grid returns
on the existing the ABC bob.
So after
looking at some
comparable implementations in
host IP, we can see
that we should see some kind of H-back
SHA256 being returned
and
entered into who the cardiac function is
argument.
So.
Through the magical power of x rays,
we find this in the
ABC code,
they're just doing standard HMX SHA256,
and after it returns, they're doing some
post-processing to do
rotate left three
on the buffer.
And
that's the only change that they've made
to the algorithm.
So this is the
crypto.
Question marks.
So
let's go back to developers, so now we
can actually go further than just the
second stage.
So we need you to spin to be able
to communicate to it that
it's eight digits and we you only showed
shoulders for symbols.
An interesting fact WPA specification
is that the design it in such a way
that there are two stages like this
in the first four digits of the pin and
two stages where in last four digits of
the pin, which doesn't
make any sense because basically, instead
of having to test 100 million pins
like you shot it digitally, you wanted to
try it for them.
You only have to 20000 pins to
actually go through your space and
brute force a pin.
And another interesting fact is that
normally most people
spin by showing the pin on the device
and asking people to enter the
pin on the access point.
And what the intent of these was actually
opposite opinions on the access points
to the console and the ask
the device to like this as a
user to enter the pin on the device,
which is a gamepad.
And that means that's
an x the opposite way, which is the usual
user as using WP spin, you can
actually brute force the off-line
without having to ask the console every
time. This is a code.
This is a quick spin on that.
So it actually only takes like a few
milliseconds to just finalize the pin.
And so, yeah, this is
just how it works.
Basically, you have these four symbols.
You just replace them with the
corresponding digits.
Yeah. And.
Ivan script, the final
and the second half is basically five,
six, seven eight, which is interesting
because it actually violates the WGA
standards, which says that last
digit should be a checksum.
So yeah,
it's actually another interesting fact
because it's like because, you know, so
that's did is a checksum issue of already
votes for the first half.
You only have to to vote for three digits
of the second half.
So it's not even 20000 combinations
to test something bigger than themselves.
Yeah, I don't know what they were
trying to do with this in the Senate, but
it doesn't make any sense.
And other people have actually exposes
that. So you can get at the big ups,
which is a tool to brute force through
your spin on sending devices is basically
just does that.
So, yeah, after that, we like
this is us two steps of the
duplex and checks.
So we just gives us information.
So, hey, is this is your side?
And this is the 32 when the bytes
keys that you need to use to connect to
the network. Great.
But obviously, that would be too easy
because we try to use a key to connect,
and that's what it's like, not usual.
And there's a big European check fails at
stage three or four,
which usually indicates that is a swing
or Santiago ism was modified.
And I mean, since we you just gave us
a key, and since we found actually
nothing to fill, Microsoft modifies this
for passengers of Broadcom chips that we
fought with, and that's satisfactory.
So that is wrong, that it's actually
using the key knowledge in the code.
So let's just look at the algorithm
again.
So to take the
packets into a dos and shake, WPA
uses a key that is called the K
pairwise transient key.
It's specific for every device connected
to the access points because it uses
Mac addresses and stuff to generate it.
And
so it basically runs a custom
SHA1 function on the
case with PMC's Joseph Passphrases.
And you gave us with like a string.
So, yeah, yeah, it's another useful
string to get for an agenda as a
predicate.
And for some reason, in our case,
explicitly whether it's just
not the same or we couldn't decrypt
what's when you sent us.
So that's who
gets so far from us this time, because
this is the only place this thing
constant is presence, pairwise
pairwise key expansion of string
constants.
So this is Cortex-A7 free code.
And what's interesting is that we started
looking at all the Broadcom firmware,
Cecille as they work and we look for the
same string. It wasn't present.
And it turns out
that looking at what other people did was
welcome. Farmers were able to see that
usually Broadcom chipsets implement
some features in Elsworth.
And the result is actually
like a function table at the beginning of
this firmware that allows you to act
overwrite functions that other
functions by software functions.
And another interesting thing is that
if you can inject where are T
as opposed to is, have this
copyright become confidential code
checked in?
So this is actually very useful because
there is just as a former code
available.
So this is for like this.
This is why it's Vienne
kids can actually point to is
Alexa, the implementation or, in
our case, the software implementation.
And so what it does is it takes a PMK,
which is a passphrase.
It puts it in reverse and then
PLF both fiscal bytes of PDK
figures.
So this is how it's dozens of EU
such a game pass.
So can you actually recognize
what this does?
Yeah.
It was fun to find out who can inspire
attacks, especially since that quiz
keeping also reversing that because we've
got some engineering parts which it took.
It was just a day or so to actually
figure this out after finding out this is
the stuff that we found the tech.
And those a what we really
turned to his.
And yes, this is actually the only steps
that are required to be able to connect
with you, to connect a PC to a view
or to kind of gamepad to a PC
and also since
to connect to be able to stick
to data. You only need the shake
and the passphrase.
You can do that. You can sniff without
any munusamy or interception stuff.
So this means we can actually just
connect like our normal game.
But so we humans intercept everything
that's going on and inspect it and see
what's actually going through the Wi-Fi
network.
So this is a very quick
analysis with Wireshark of
stuff that's passing through that
the game, but the console and to the
consoles, the gamepads.
So as you can see, the trees using IP
on this Wi-Fi network, which is
interesting so that only two devices
so they use one nine
eight two one six eight one 10 for
you and one for the gamepad.
And potentially you haven't seen that in
use yet, but point four point twelve for
the second gamepad, which
well, it's interesting because they say
it supports it, but we haven't actually
found anything that supports it yet.
So
there is only one thing going from
so gamepad to the console itself.
Rs forty or fifty one
two two it's sending
back one packet
is sending one hundred and eighty packets
to second with stuff that was
actually fairly easy to work on as input
data because if you press a button just
changes
more interesting stuff like Ports
520 and five 21,
which I should probably be doing oju data
because was it just a lot of data is
going only one way, and
the gas protocol is actually
it's actually the same number of packets
on both sides. So probably just replay
response to to confirm that.
But just looking at the extenders.
So there's a surprise for us that there
are a lot of articles on the internet
saying, Oh, so we're using Miracast to
connect to the gamepad and the fact that
so we re assuming we just find something
standard and be able to just implement
Baracus analytics.
So that's well, I don't know where
I got this idea, but this is actually
completely wrong because there is no
Baracus shoes anywhere in the EU.
It's actually only using.
It's using concrete features and
protocols over the whitefella.
So everything is based
on UDP, which have one protocol for
video, one for oju, one for input data
and the fourth protocol, which is just a
simple RTC thing.
From that, we use the gamepad
for Suffolk turned off the top
of the screen or
adjust screen backlight.
Oh, all that sends you said FC
commands or something like that.
It's a very easy protocol.
You an input also very simple.
It took us a few weeks to get
fairly good implementations.
Garcia to Congress.
We actually liked figuring out input
data, and lots of it was straightforward.
VIDEO actually was way more complicated,
and that's why we didn't actually present
this year. But on this issue, it took us
actually far too long to get video
working.
This is actually a very recent
development two months ago.
VIDEO With six not working
for us, for some reason we were able to
decode stuff without we encode stuff.
So yeah, uh,
this is what happens basically this year.
Like, we just stare that idea for a few
months and magic happens.
So the gameplay is actually fairly
complex, like you could see on the
on the PCB that you actually have
principles.
It's actually got more chips on it and
you have this principle, which is a DLC
the way up, which is now known.
It's a custom system on chip, so it's no
documentation about it.
We are assuming that it's made by MCA
Chips Corporation, which are like working
with Sentinel for some stuff.
It has four gigabytes of ram and chip,
and it has a custom issue 64 decoder
for video.
It communicates via is always
a Wi-Fi chipset, which is fairly standard
for embedded devices, and it's
nice to see both for few stuff against
the controller. Audio control and NFC
is asked you out for the other stuff.
And most people it doesn't has its
to communicate with the flash,
which contains the server codes, and
the UAC, which is another super
which estimates it
could be added SIPTU, which is not very
powerful.
And this if you indoors
basically correcting all the inputs and
gamepads and testing a big burp of
data to see TLC, which contains
the value of all the inputs.
So the US see itself as
one of the survivors and another I2C bus,
which are connected to all the input
devices on the console, including, by
the way, the extension bolts, which is at
the bottom of the gamepads, which
which currently has no use except
provide a way for Nintendo to like
do to put a gamepad in debug
mode. Well, not in debug mode that in
fact repair mode for factory reasons,
we guess.
On The View, we have another system on
Chip, which is the drage, which is
extremely similar to the DRC.
So it's another custom system of tiered
system. Chip was about the same amount of
RAM and it endless communication
between the DLC and the Wii U.
And this, which
communicates with the rest of the console
of the USB package, is exposed as a USB
device to the WHO you operating system.
And obviously it passes
webcam microphone input and comments.
And it also gets video and loads of data
to work with you.
It's configured as not pathetic,
just like the chip
and Typekit who gets video
data for that.
So let's look at the firmware off of
the gamepad.
So it's all going to is mostly concerned
this is Elvis Rebrov, which is
the firmware for the online CPU.
It's based on these false Makasi Free,
which is a Japanese with operating
system, which is fun because that means
there is no documentation in English,
obviously.
But but it's interesting
to me as a Microtonal 4.0 specification,
which is basically an API for its
operating system, which is inverted by
like a few. We also are facing so as far
as I know, like our services that can
open source or to us printing it.
And this specification actually like
is very well documented.
So it runs about 50 background
tasks on this review, which
is like, what is it doing
this 50 50 processes and this
thing?
There is no preemption.
Select competitive stuff and
it's very interactive and that it gets
most of the communication was out of our
interface.
It is for us, it's not shipped with civic
symbol, unfortunately, and that some of
the parts that we use that people talks
about on day one, that it's taken these
fathers because have these assets
statements just about everyone's a
code base and access
your assets statements if it's fair as
just shows up.
So is a safari.
There is a source code.
So from that, we can actually get
like a very simple idea of what's
select code is organized.
So they are forming the base of modules
drivers, which are basically two modules,
and we're in communication with other
services which are all network
and king parts. Actually UDP servers and
UDP clients and managers, which
are what basically what
the services talk to to do stuff in
the contract states and communicate with
drivers and send data to services and
stuff like that.
And also applications just for all of the
codes that actually like needs to run as
a game pad in space stuff and a gate pad
and do stuff with controls and
safeguards. For example, like a TV remote
app is an application.
Another thing that will surprise us
actually into new in this form is using
a lot of different types of thus
communication stuff, sometimes
completely redundant and sometimes which
don't make any sense. So if you have the
usual data, choose and values in
smartphones and music since it's
also have these systems, which is events
which are called cross process
and lets you follow the case of using
a memory pool was only one brokerages
stood to do in the task synchronization.
So basically, is this memory pool?
Is it consistently one block
of memory? Social class takes most things
a block of Fillory, and those are
process. Try to take another block.
If we just wait until there is a block
that is written at all.
So they actually use that to do basically
a mutex, which
I said all to this and was still doing
what they were doing. It's probably just
an intern writing some code or something
that, yeah, says
I wasn't doing.
So as the other part of the civil
disputes of the parties is
the US is the estimated CPI, which is
used to handle inputs
423.
There isn't an official estimate
idea of plug in, but we're going to get
through what one is related to as well.
And this CPU, as I said, communicates
with the online codes.
Vice Verse five is
connected to all the devices,
he says have an interesting
point. All the settings for the gamepad
are stored in F1, so if
the if the main CPU actually
wants to read some settings it
has to send, the command is a UIC
to read stuff from the US F1
and sends us the sensitive back of a
spine just provided to
it.
And they also they also don't provide
that high level stuff to access is
settings. As your secretary just sends
back to the old brother settings, which
you got eight
eight kilobytes, something as
X game is a game, but just passes it all
every time.
So video swimming.
I'm going to hit shuffle, talk about
that.
Right, so as you heard, we didn't have
video streaming to the device actually
working until pretty recently.
And this
was because
for unknown reasons at the time,
but to in order to figure out, we
tried to look at the
packets that were being sent in the
stream and compare it to existing
protocols.
And we didn't find anything that looked
really similar.
So we just went to x
rays and passing
the stuff ourselves to figure out what it
is.
And we assume that carries a
six four, partly because the drug has
some strings for debugging.
That suggests that.
And we tried to prove that it was
actually true, even though we
couldn't decoded as H.264.
So after reverse engineering
the firmware on both sides of the
duration of the RC, we're able to
find out the meaning of the entire header
of the V stream format.
So the main important parts are
the frame begin
and end chunk and flags,
and the timestamp
played an important part
and.
So just from staring
at data that was going over Wi-Fi, we
figured out the bits
in the header and how they corresponded
to frames beginning
ending and.
Chunks which are kind of like H264 slices
ending within a frame being transmitted
and and you
can also see that the
time stamp is being incremented at.
We had to see here, but being committed
at a normal rate, which matches the fifty
nine point nine four hertz, which is
advertised.
So.
We figured out the entire header and what
it meant, but it still didn't help us
decode the data was being transferred
in the payload.
So instead of continuing to look at how
the gamepad was receiving video
from the Wii U.
We looked at how the WI was
decoding videos you seem to be receiving
from the camera on the gamepad.
And this turned out to actually be pretty
interesting because it's not
decoded on A.
H. Even though the DIRIGE has
the hardware capability to do that,
it's passed all the way up the stack
through isyu on
the Starbuck onto the PowerPC,
and then it's given to give you to
decode.
And the drage
plays a part in this by
DeCaro parsing of the extreme
format and converting into UV-C.
And as part of that process, it processes
prefixes headers on
the reassembled data frames
and the headers
turn out to actually match
H.264.
Now, encapsulation headers, so
it provides the decoder
parameters which are required to pass the
compressed bitstream that follows.
And in order to determine
the actual settings that were being used,
we had to reverse the
video library on the PowerPC
and determine the meaning
of all the flags that would be used to
decompress the bit stream.
And this was actually a lot easier than I
hoped after I reached this point,
because they have this huge block of
print statements for debugging, which
lists the entire structure of the
decoding.
And as a bonus, it maps
nearly one to one to AMD's open
video decode library, which you can find
on GitHub.
And so certain
bit fields were really easy to
use from this.
So this actually allowed us to decode the
video stream that was in the payload,
and as a side note, we noticed
that the video stream didn't have any now
encapsulation, which
are basically just.
Little markers on the beginning of frames
to determine where the data starts,
and it also
escapes certain bytes within the stream
so that you don't accidentally be in
a frame when you don't mean to.
And so we could actually
play a video
here, stick psychic.
So this is a video of the gamepad
to where you connection being sniffed
while it was running the ME
Avatar creation app
and
after capturing it on PC, we could
strip the V stream format
into normal H.264,
apply the correct decoding stuff and
it worked correctly.
So that's pretty awesome.
And actually, we
did that because we were hoping if we
could change the resolution parameters on
the stream, then it would also work for
the gamepad video.
And it did end up working
to decode that as well.
So now we can decode videos, and
because we have those parameters for the
decoder, we can do the same for the
encoder.
And we were we
made an app that streams video to
the gamepad, but it wouldn't work.
We could get like maybe three quarters of
a frame displayed on the gamepad at
most, and it would just freeze
after that.
So in order to debug this,
we had to actually change code on the
flash of the DRC
in order to add debugging back
to the host PC over a UDP.
So we reused some of the
migration code to
open UDP UDP port, and
we're just right to it.
So it's actually pretty easy to
implement, and it allowed us to get
all the assert statements that in the
code and
some things like the timing registers
and other MMO things that looked
interesting.
And after doing this, we found that
the problem was related to
the extreme timestamp in the header
not matching the timestamp, which the BBC
was reading from some internal
register.
And this confused just for a while
because the DRC actually has
audio, St George have about
five timing registers which appear
to match to the same
clock value, and
the one that was being read for the V
string parsing is not written
to ever. So we assumed it was just being
started from zero boot or something.
And yeah, yeah, OK.
So, yeah, after some time,
like we always friends, it's like we're
stuck on this phone for a while because
we just had this clock,
which was incriminating and we couldn't
figure out why this was initialized and
how it was actually precise.
And then one day I went and we noticed,
Hey, I just rebooted the device,
and the value individual Typekit were
just so weak is convincing
as the same way to says the value that
is consistent keeps you into forgetting
to end times and you start to make it's
it's like, is it is basically
incriminating as a coke trades, even when
gamepads pull it off.
And actually, even when the game was
a great, the new power was that we
removed the.
What's?
What's this doing a cold OK and actually
keeping Quincy that tons of battery as we
move? It doesn't make any sense.
And with seconds, that's like
a few days since with me, I say,
actually, we just disconnect
Zach's, especially the computer execs.
The Sun just goes back to zero.
And so it turns out that there is a
midwife, a feature that nobody actually
uses setsuko timing, synchronization,
function, TSF.
And what happens is that when you send
the Nexus One beacon
that you access, you access phones
usually extends beacons to say, Hey,
I mean, access points, you can connect
to me. That's what you see when you scan
for life networks on the network.
So
this is access when beacons actually
contain a timestamp, which is 64
bits in microseconds and.
It that this is meant to
be for stations, access for
clients to synchronize a clock on
basically the same value.
It says that this is actually what's
being used for this video and audio
synchronization.
So we actually didn't even know about
this feature before finding out about
anything else. Oh, it was a day to the
oh, it was a dating site as a time for
video and audio.
But yeah, it would be too simple if we
could just use this time same value and
for a video swim.
So I mean, this was actually
no way to get decent value from New
Zealand. It's only available if you
like connected in wi fi,
adult of something and then only
on a specified system.
But it turns out that most of the drivers
super the way I could send
a way to provide it with the TSF
and so we can just export
is to discuss this file and
give access to it to users and.
But if it turns out that's in,
it tells of issues, so on voting
devices, we actually figure
out that the TSF actually drift slowly
after one second is first satisfactory
in quantity like one second
plus 10 microseconds.
So it looks like there's a clock in the
voting is actually go a bit faster than
real time, and
it's actually even worse than the
afterwards devices because it's not only
one second microsecond, it's actually one
whole milliseconds.
So that is going way too fast,
and the timing window
on the gamepad is actually only one
milliseconds.
So it's been set to one second clock
has completely drifted away from
the normal values, and we can't actually
send it to frames anymore because
sometimes that just doesn't match
anymore.
And yeah, I mean, on Intel devices that
just don't provide anything for TSF
for me because nobody uses it anyway.
So it's just returns minus one every
time. I'm not even sure if it's if there
is a way to get it back from the
hardware.
So this is one of the major problem we
have is until the devices at the moment
ends with Apple devices,
we couldn't actually find an explanation
of what was going on.
When you actually ask to
first try for developers and you see that
all this? Oh, it's not a bug.
It's supposed to be like that,
which doesn't make any sense if it's a
timestamp which is counted
in microseconds and it doesn't count
microseconds.
I mean, it's it's just a bug, but
it's probably don't want to admit it was
often the way to fix it.
So with that, actually, we
were able to start sending some video
data and get it properly a game pad
because it was actually Zong wishing
we could be blocking us and a few that
out about two months ago, basically.
But even when you are using synthetic
fabrics, as we got from decoding stuff,
we still have a lot of video corruption
happening and actually even crushes in
some like because
we see other people can code and stealing
killed showed us when the decoder
actually fades, decode the frame and the
logic of the face to decode the
framework. Two time interval is a game,
embeds are concrete, blocks have been
crashes, so we have too many reboots.
And yeah, when using proper encoding
politics, it's like it happens.
And so we just couldn't figure
out what's happened.
And we
we felt like three years of features of
what's causing this issue and
just disabled it.
So now it's not crashing anymore.
But I'll see lots of video decoding
issues such you probably since the Jim.
So is this actually a screenshot for
was the first time we got
video work shows a game pass.
As you can see, we have a small grid
is because we wanted to be able to debug
the value of each H.264
could work.
So six and six and pixel is
that way. We can actually see
where it's where precisely it's failing
and why is figuring that just
checking the stream with another decoder?
So what's the current coincidence of the
project of today?
We actually now developed a small
was given library to be able to look
for every developer to use the gamepad in
PC applications currently only works
on Linux, mostly because
on Windows, we just started
actually just starting to figure out how
to make way for access
points of fact. That's because it's
because patching the drivers and patching
the Windows whiteface stack, which is, of
course, source, but
it works on works and it should work on
windows at some point.
It comes with nice demos and
documentation about the API,
but also about.
So it was and that's what protocols and
oh to use like a reverse engineering
scripts.
One big caveat of it is that it doesn't
end all of the pairing stuff in network
stuff. It's a completely separate.
It assumes that the network is already
working, that you're connected and you
have separate IP addresses and stuff like
that. So it's basically
nothing. Actually, we haven't tweeted
anything to any of that yet.
So if you have any patches
and software to work on,
I think you should be able to access will
get just all websites and it has
all the documentation and stuff that was
over.
So you had kind of it's it's an alpha
version, it's extremely buggy, is
not meant for end users,
even though getting the network right
isn't easy and swabbing means take lots
of time issues until the that find so
that. But it's still a good prototype,
and we're hoping that we can actually
make it a very useful thing
and that some people will actually help
us continue to make it user friendly and
easy to use.
So now just
some simple example codes.
This is if you look how it looks, you
just create a streaming object.
You started and you can push with your
friends or your samples, for
example, to get input data.
And we have a nice feature which enables
system input filter, which we're just
basically using you input API,
which is part of Phenix, to send
all these ID controls from the gamepads,
direct feedback to connections in the
works, which also has to translate
that to invade a real
joystick plugged into a computer
so we can actually like do in user land
and create a proper gamepad so we don't
have to modify anything new application
if it already can use a normal joystick
or normal gamepad.
It should just work automatically.
Sure. Just fine, of course, which is
nice.
Also in this, who is just a few minor
stuff, obviously quiet patches for civil
projects. So x264 because
we had to disable including
switches and safeguards, OCP pitfalls
of qualifying modifications.
Make make sure it is who we are then,
which is see Linux drivers that
handles Wi-Fi.
Basically, we read this also
some other scripts for free movies, a
reverse engineering if anyone is
interested in as Dante's,
will you again. But further,
we also have a very simple
do we to perceive stiffer, which will
like intercept communication
between you and your gamepads.
And we'll try to show
the video stream as it goes live.
Unfortunately, because of how to
video to swim for it isn't
because of the videos to ThoughtWorks.
It doesn't entail packets as well.
And you often have packets for some
Wi-Fi, especially when you send a
few megabits per second of data.
So it's just a story.
Basically, it's utterly usable for
anything, but I just wanted to try it
in wealthier Wireshark plug ins and
worse patches to be able to use
a data.
So now it's time for them.
If it actually works,
because I mean, you know, about
everything else never works, but this
whole celebrity movie using Wi-Fi in
action in a building with 4000
people also using Wi-Fi.
So yeah, see if it works.
So just to start
this program.
OK, so now it's just
for once again.
If it works to actually show a program.
Yeah. So this is just a simple demo
which allows us to basically do touch
screen and tool stuff.
I don't try to with them just
let its touch screen doing
stuff, they're easy.
And we have more fun because
in that clip, we just started.
So.
It's just we sort of gamepads,
this is one of the bugs like it should
be, but what if you take things in your
video service? It doesn't.
That's what.
Yeah. So does anyone recognize this game?
So this is actually running on the
dolphin, everybody told the ABC.
So this is like imitating a game to a 9pc
and sending the video data to the game
pads and.
Straightens for good sometimes just
crashes, but it's not without watching
of football, you can just restart, it's
like. Yeah, I mean this, I said, it's
an alpha
actually being beaten to death.
Does having this latency is very good
tells you video artifacts still.
I mean, but it's mainly
working, which is very nice.
I mean, we wouldn't
expect such a good visual latency and
stuff like that, but it turns out it's
just OK.
So yeah, this was this was a demo.
Thank you.
So if we can just good exercise, maybe
just have, I think, one outside.
Oh, yeah, this is just another screenshot
of as a fallback.
This is Final Fantasy seven on the PS4,
the rate of rendering and some who you
gamepads.
But yeah.
So this is just back basically to
the list of stuff that might
be interesting to do, so obviously
actually find it also with the policies I
already talked about that windows in
those exports.
There's also one thing that would be nice
is Android ports not to be able to
stream video from an Android device to a
gamepad. That's the other way around.
Excuse the Android device, excuse
entering device and connected to us
to leave and use it as a
custom game. But it's not like official
Nintendo stuff.
So I have I have with me
on Nvidia Shield.
I love you and you are a few of those.
And we just showed is basically an
Android based game console, which has a
touchscreen and physical buttons
interface is based on Android.
So what if I can't project this to try to
connect that to you?
And you still the game? But
one thing that would also be possible?
Thank you.
Let's fields will also be possible,
actually is a physician
for the vehicles can do so
fetch stream games,
succession for two, a veto even obviously
over the internet.
And we actually think that this might be
possible between the EU and the EU
gamepads using some kind of proxy system.
We haven't actually tested that, but we
don't see anything that would make it
impossible.
So if anyone just wants to use that to
try it, this would be actually a very
cool feature because we
are fixing video encoding of just
resolution it with you.
But it's also about especially
if you just look as a screen instead
of a camera.
Cancer screen is free of uses a lot of
video artifacts.
Yeah, I mean, this is just going to be
fixed by working with x64 people and
trying to figure out what's not supported
by the decoder and what we're doing wrong
and safari that's
just off shore this stuff here.
If you've got a data sheet for the
justice system on chip, we obviously
interested because we see that basically
all this always custom, you have no idea
how it works accidentally.
We haven't found anything similar to the
H.264 codec that they use,
that it's like, but you cannot budget
slapping and feature sets and stuff like
that. So I
wonder if it's actually really or if it's
just something that we haven't found
the food we can actually be upgrading of
this network, which is interesting
because that means you can like
if you have if you have a device that can
connect against the gamepad, you can
actually do modifications to game that's
code and it's not signed or anything.
So you can just do whatever you want and
it. So we have I have a project
to just try to add
a simple obviously stuff in this gamepad
to be able to read and write memory and
change some codes and be able to
experience physically and then maybe work
on like firmware replacements
and stuff like that because it's like
it's a device running at winning arm
codes, actually. There is no reason,
which means you shouldn't be able to run
your own code on it, and you are shoving
the idea of a code extension for devices
to put on this game. But just try it out.
It's just like a an actual sports
explosive action on support
as buttons you see.
So that's a talk.
Thank you very much for attending, and if
you have any questions, I think
we'll be taking questions.
Yeah, we will.
We will be taking questions if you have
questions, just line up behind the
microphones there in the alleys.
So we have a question here on the far
left.
Yeah, hi.
How much of a chance does Nintendo have
at blocking this stuff?
Those are probably kids are surfing.
Is the flash not being encrypted
devices?
I think it's not something you to say you
can choose. So even if they start
actually fixing stuff can see, like
reverse engineered all the communications
into it again, if
they also don't have much of a point to
fix that because it sounds like it allows
people to pirate games or anything like
that. So we're not actually
sure what's what Nintendo's reaction is
going to be. We obviously just let it
go and let people use a gamepad for
whatever they want.
But if this sort of fix stuff obviously
will try to adapt and fix our stuff as
well,
I'm specifically worried that
if you do something through the firmware,
it will stop working after
you update.
So we've got this up.
It's actually fairly innocuous, as it can
detect for modifications because
there is no signature on if you would
have to actually like as a firmware or
something like that. And you can fake
Xbox ISO.
So do we have another question, maybe
from the Internet Signal, Angel?
Yes.
Mindstorms asking
what CPA there is in the game
and especially the clock frequency?
What do you mean? Like, what have you?
Because our principles was basically one
nine one Cortex-A15 one
estimate, but
I don't know what the exact question is,
which didn't in the Senate.
So sorry
for continuance.
Yeah. Oh yeah.
We don't know a lot of details about all
this abuse because it's just another way.
Basically, like the
Armco is a standard on call
basis as a system and ship is completely
custom, so we don't have a lot of
documentation about that.
So, so fact frequency and safeguards,
we have no idea yet.
Yeah, OK. Signal Angel's feel free if you
have another question that's
also for.
OK, well, so thank you.