So, I have already been introduced
My name is Stefan Widmann
and maybe I can have my slides?
Hello, my slides?
*laughter*
Ich bin am VGA drauf.
[hums]
Yeah, um...
[mumbling]
Okay, so while we're waiting until my slides appear somehow.
Who has seen the incredible talk about
hacking the VoIP phones from Cisco, last year
either live or on video?
Okay some of you.
When we think about this talk
The Cisco VoIP phones have an embeddd
Linux operating system, but they did not only
have to deal with the linux OS, but also with
the firmware of the DSP.
So I want to tell you there's not only one system,
but several systems. Several sub-systems containing firmware
Slides would be nice, we can start without slides, there's no problem.
So what are we going to talk about today?
First we are going to talk about motivation, why should we do firmware analysis.
Then we need to be able to do it, so we have some prerequisites to bring with us.
Then we'll dig deep into the topics, We will try to look at how we obtain a firmware,
how can we analyze it, and how can we modify it.
Hmm. Angel: We are sorry for the brief hiccup, we're working on that. It's the second talk. I'm sorry.
[speaks german]
Hmm
Okay, so we can do without slides, it's okay.
Let's start with the motivation, why do we want to do firmware analysis.
When talking to my lawyer I learned that I had to clean up 90% of my motivation slide.
And left is, you can do it if you want to gain interoperability.
*laughter*
You can do it if you want to get rid of errors, and the manufacturer
does not want to or is unable. And one interesting point under
discussion is, what about forensics, taking a look in those thousands of devices
in everyday life. Do they only do what they are supposed to do?
Herald: Yes, we are still hunting for a Video Angel to sort this little problem out. We should
have one here within 1 minute. Again, very sorry.
We now have Nick Farr on stage, our certified Powerpoint specialist.
*applause*
It can only take minutes now.
Maybe we can continue. So I will just tell you something about prerequisites you
should bring when starting to analyse You should at least have a good knowledge
of embedded system architecture. You should have dealt with peripherals,
bus interfaces and so on. You should be able to read
and write assembler. Some might say:
I have a very good decompiler, which is fine.
If it works for you, okay, but don't rely on the availability of a decompiler
for the architecture you're going to be working on
Especially if you are going to work low down in the register stuff.
And in my opinion, a decompiler output will confuse you more than help you.
You will go to disassemble maybe, C runtime libraries, optimized to be as small as possible
That can be really hard in decompiler output.
If you want to practice how embedded systems are working, then it might be a good idea to fetch
your arduino or whatever. You write some little C code, handling some hardware stuff.
Then you just compile it and take a look, what the disassembly looks like.
Very nice to have is a device reader or programmer, like galib. The problem is they are expensive.
If you think we are going to do firmware analysis, it may be a valuable investment for your hackerspace.
And last but not least, what you need most is time.
Time, time, time. It may take hours, days without any progress, so please be patient.
*laughter*
Any volunteers to make up some slides here?
I swear, it worked perfectly okay with my external monitor.
Yes? [illegible]
So I'll have to fetch my USB stick, wait a moment.
No problem, we're flexible. [whistling]
So is there anyone who knows their way around a computer around here?
Herald: While we figure that out, it might be a good possibility to remind you all
that we are still looking for some Angels. You could do video angels, which are in
high demand right now. Or you could just do any other work you'd like to.
You can do one or two shifts, that's fine. It would be greatly appreciated
because we require volunteer work for this event. Also if you brought any beverages in here,
it'd be awesome if you could take them out with you. And put them into the little
storage cases located all around the building. Trust me when we are finished,
you'll be able to do this announcement. Are we good? No, not really.
*laughter*
It looks good on the laptop.
You want to give a quick intro to the new Ubuntu desktop?
Because I did not get that at all.
Yeah, mirror displays man.
4 zur 3 folien.
Now we are making progress
*applause*
Enjoy the talk!
Okay, perfect, now with slides.
One small announcement, there will be 5 more minutes of extra talk at the end.
We'll do that. So just please ignore the yellow bars.
*laughter*
Not really no.
High level devices. Big complexity. YES!
Perfect, thank you, without yellow bars, thank you.
Okay. So we already talked about prerequisites, so now we're going deep into the topics.
First we need to obtain a firmware. We will go from non-invasive to invasive.
Because first thing we want to try is getting the firmware without opening the device.
We will first try to download a plain binary from the manufacturer.
or maybe someone else have extracted a binary and placed it on the internet.
You can try to download a bootdisk, USB, CD-ROM, bootimage, whatever the manuf. provides.
and extract it using, for example, WinRAR on windows or just mount it on linux.
Search for files named .bin, .hex, .s19, .mot. like motorola, .rom or .raw.
Most times binary, that means .bin, .rom or .raw files are already real binary files.
Non-binary files should be converted to bin files, e.g with converters like hex2bin.
If this doesn't work out, maybe we get an updater from the manufacturer
Normally they're .exe files built for windows.
There are different updater types. First the self-extracting archives.
It might be an installer too, like Installshield or whatever.
It might be an updater, simple .exe file without any installation, just containing the image.
It might be an updater that is downloading an image, or it might be some of the others,
but packed with an executable packer like UPX or PECompact.
Let's go a little bit into detail.
So if it's a self-extracting archive search for signatures like RARSFX or PK.
You can unpack them, e.g. if you rename a PK containg file to .zip you can unzip it.
If it is an installer, like Installshield, there are special unpackers, but the problem is they are
very hard to use and extremely version specific. It might work, it might not.
The best way is to just let it install and search in the installed files
for a plain image or updater. If it's an updater containing the firmware image,
we can search for the image in the executable using your favorite hex-editor.
Maybe the updater is writing the data to a file, a temporary file in most cases, and deleting
it after usage. You can use ProcessMonitor which is like strace but on Windows
and you can take a look at what files are written to disk can try to capture it
before it's deleted. Maybe the updater is just checking your device, so its just
a little downloader. Checking your device type, take a look on the ftp-site of the
manufacturer and is downloading an image if there's one available.
So if it's downloading the image to a file, use ProcessMonitor again.
If it's just downloading to RAM, you have to go for a debugger, and dump it from memory.
If you have a packed updater, which of course is only done to save size.
If it's standard UPX, you can download UPX and use UPX -d to unpack it.
Sometimes the manufacturer violate the license of UPX and modify UPX by removing
vital file information to make it un-depackable. So you would need a special unpacker.
Other executable packers are most times designed not to be uncompressed.
So you would need special unpackers too.
One challenge that awaits us is, maybe the updaters contain compressed images.
They are normally unpacked before the image is written to the device, so we can just watch
the process memory with a debugger and dump it.
What's a bit more challenging is when the firmware is sent compressed to the device.
So we have to use invasive techniques we will talk about later.
It's a good idea to get a sniffer ready when you first connect your device to your PC.
Maybe the favourite bloatware coming with the device wants to update it instantly.
What can you do to sniff the transfers? On Windows XP and I'm sorry it's only XP,
there's TraceSPTI, a fantastic tool tracing SPTI SCSI PassThrough Interface.
So you might think SCSI? I do not have any SCSI devices, but very much communication is done using
this protocol on Windows. to identify S/ATA USB devices, especially if they are ATAPI.
On the linux side you might use Wireshark to trace the communication, because Wireshark
on linux can trace and sniff USB. There are various other tools like Bushound and so on
to watch communication on buses. But the problem is they are normally very expensive.
A problem you'll have if you're trying to sniff the update transfer and reconstruct the image is
that it's like a puzzle. You don't know how to build the image, and if you're doing it right or not.
If we do not have a firmware yet, it might get invasive now.
We'll search for serial interfaces, sometimes they are accesible without opening the device,
sometimes not. Do we have an embedded linux system? Yes, we can search for a serial console.
Maybe we have to use JTAG, there was a very good talk on 27C3 about JTAG, serial flash and so on,
so I've included a link here.
So, still no firmware? Get your screwdriver, let's void warranties.
We open the device and we search for memory devices on the PCB.
If you have a very old device, maybe you'll encounter EPROMS or even PROMS, 27-somethings
If it's a little bit newer, you might see EEPROMS and flash. 28, 29, 39, 49 something and
and the big flash devices with 48-pins for example with various other names.
Very nice to see is that serial flashes, those 8-pin devices 25..., sometimes 24...
are more and more becoming the standard. They are easy to de-solder, easy to re-solder
and there are very cheap readers and programmers available. But please, even if some say we can
do it in system without desoldering the chip, please don't do it. It can lead to very big problems.
To make it a little bit harder, firmware can be contained in chip-internal memories.
You can try to use proprietary programming interfaces to read the firmware, of course JTAG.
Some devices do have bootloaders in a mask ROM. You can try to use them.
If none of these approaches succeed, you can try microprobing.
There was a talk on last years congress about low-cost chip microprobing, I've
included a link here. So just for matter of completeness I've mentioned CPLDs and FPGAs.
You know CPLDs are built up using internal EEPROMs.
FPGAs, Field Programmable Gate Arrays have internal SRAM and an external serial configuration flash.
Some years ago they were marketed as being reverse-engineer proof, okay. Yeah, maybe.
There's a talk tomorrow, same time I think, in Saal 2, about taking a closer look at FPGAs.
Yeah, congratulations, we've done it, we have our firmware, perfect.
So what's next, now we have to analyze it.
The problem is what processor is used. We don't know which disassembler to use.
So we are searching the web for any datasheets, can we get any information.
Can we find out what processor is in use? The problem is in many cases you won't get the datasheets.
The manufacturer says, you buy 1 million devices a year, and you sign an NDA, you get the datasheets.
Now you have to be really patient, now it gets to trial and error, trying different disassemblers.
You can use specific disassemblers, they are only built for one architecture.
You can use a very good tool, the Interactive Disassembler, IDA. There's a freeware version.
I've included a link in the link section of this talk, but the freeware only has a little set of architectures.
If you want the full set, it gets very expensive. But there is a new tool that I really like.
It's ODA, the Online Disassembler, supporting thirty something architectures, and it's free.
You can upload binary files, you can upload code, and try different architectures,
and find out what might be the correct one, and we'll do that now.
So I've prepared some binary code.
I know which architecture this has been written for, because I did it.
I put this code to Online Disassembler, and I chose different architectures,
and now lets look at what the disassembly looks like.
Let's first start with former Hitachi, now Renesas, H8S. I hope you can read it.
Take some time and please raise your hand if you think this is valid disassembly and
we have found our architecture.
I see one hand.
Okay, I have to disappoint you, I'm sorry, it's not valid disassembly, we can see it in the second line.
The disassembler was not able to disassemble the data and it's just an undefined instruction.
There are several .word's in the code. It's not H8S.
Let's try MIPS. Again take some time and raise your hand if you think that's valid.
*laughter* again?
It's invalid too. We can see it in the second line, because there's a dword that's not disassembled.
What about Panasonic MN103 family? The same hand again? Oh I see another hand.
Ok, several hands now. Yeah, OK, thank you.
So the problem is, it's not valid. I have to disappoint you.
The problem is in this case, it looks really good and you have to dig deeper.
You will have to look at, are all subroutines correct. Do they make sense?
Are there subroutine calls at all and so on.
And you will see something strange. Ok last try.
What about Texas Instruments MSP430?
And again, please raise your hands.
Okay? Yeah, this time it is MSP430!
We have found our architecture, perfect, eureka, bingo, we have it.
But what's next? The offset in the file, of the firmware file we loaded is often
not the offset in address space. This is no real problem when the architecture is
using relative adressing. Relative adressing means we have register content and whatever
we want to access is based on some registers content. Location independent code.
But we have a big problem when absolute adressing is being used, and even architectures
supporting relative adressing do have some absolute adressing, somewhere on some accesses.
We would not know, where's the entry point. Where should we start?
Interrupt vectors might be decoded completely wrong, subroutine calls do not make any sense.
They go to [addresses] outside of our firmware for example, or in the middle of instructions.
So the load offset has to be found.
I'll now show a method I call "call distance search".
We will select closely located subroutine adresses and we'll have to decide either to use
preceding return instructions in front of the subroutines, or the start of the function
entry sequence. We build a search string containing wildcards, and then we search.
Now we'll do that together, I've prepared an example.
This is 8051 code. The 8051 core is very old, it's an 8-bit controller, but it's still widely used in the field
because it's cheap as dirt and you can implement it wherever you want.
In the left column we see the addresses of our example, from 0x00 to 0x13 hex.
We see four subroutines, with the first being the root subroutine, calling the other three subroutines.
We can see the first call to 0x100 is outside our example, we do not have 0x100 in this example.
So what we do is take the three subroutine adresses and sort them.
So we're getting 0x100, 0x103, and 0x107. We calculate the difference to figure out the length of the subroutines.
We get 3 bytes and 4 bytes. Now we look at how subroutines are built in this specific architecture.
On x86 you will mostly find it, not on the 64-bit platforms, but on the 32-bit and 16-bit platforms,
You will find a stack-frame entry sequence in every function, like push bp or push ebp 0x55
So you can trigger on that one.
On 8051 it's not possible. Take a look at address 0x0A It's 0xE0.
Take a look at address 0x0D , it's 44, and 0x11 is 7B.
The are not equal, it does not help us.
So we look at the preceding returns and yes there are returns in front
of every subroutine. So we take the 0x22 [ret] as our anchor.
Our search string will look like this; We start with the 0x22, we have a
subroutine with a length of 3 bytes, so we have 0x22 [ret], two wildcards and
again a return. The second part of the search string encodes the second
subroutine with 4 bytes. So we have wildcard, wildcard, wildcard and again a return [0x22]
In this simple example we get only one hit, perfect. We get a hit at address 0x09.
But we do not want the address of the return, we want the address of the subroutine,
so we are not using the 0x09, we are using the 0x0A.
What we do is we take the original destination address 0x0100, we subtract 0x0A
and we get the base address of our code example, which is 0xF6
If we apply this newly found out load offset to the code and we adjust the offset
starting now at 0x00F6 in the left column we see that all three subroutines now match.
The call to 0x0100, the call to 0x0107 and the call to 0x0103.
Ok, I think this was hard, so let's repeat what we have already done.
So we have obtained our image, we have successfully found the processor architecture,
we have found a disassembler to disassemble the firmware,
and we have hopefully found the original load offset. So what's next
Maybe the question arises, is there additional firmware in this device?
I see jumps and calls outside of firmware we already know, although we have adjusted
the load offset. Is it chip internal? We can see it on the figure, maybe
we have only firmware part A. And maybe it's using a library or chip internal part B.
So we will have to see what we can do using a modification of the firmware.
Now having done that, we can start with normal reverse engineering of the code.
We search for strings, we search for references to the strings,
but as we are in a very low end embedded system, maybe we can search for very specialized,
data references and operands. Search for USB descriptor fields, you have extracted with /bin/lsusb
Take a look for USB magics like USBC and USBS, you know these two dwords are used in
usb communications. Take a look for IDE, SATA and ATAPI ID strings, saying
"I'm a OCZ SSD device" for instance. When you've sniffed the device communication
you've already found some typical datablocks. You can try to find them [in the binary]
Last, but not least, maybe the device provides some error codes, and you can search for strings,
or for operands in the opcodes.
It's very interesting to find hidden firmware update sequences, because
they would allow non-invasive modifications. For example search for chip erase and
programming commands, you can take the appropriate commands from the datasheet
if there's any external memory device available. We've done it, we have analysed it and we've
learnt a lot about the device. Now we are going to modify it.
First, we have to think about, If we are going to modify the firmware, we have
to prepare to brick our device.
Manufacturers implement several integrity checks, and why do they do that?
They do it because firmware is stored to flash, which is prone to aging, especially if heat is involved.
So they do checksums. There are softwarebased checksum calculations, CRC for example.
There are even hardwarebased checksums where some HW peripheral will do the job for us.
So what you see in the code is maybe the start offset, the end offset, and if you're lucky the polynomial.
It might be hardcoded in the peripheral too, so you won't see anything.
It can be a combination of both, being done only on startup or cyclically in the background.
What we have to do to modify the firmware is either correct those checksums,
or we have to patch those checksum algorithms not to trigger.
What are the goals of our modification, of course we heard it in our motivation section.
We are about to correct errors, and maybe the errors are contained in another part of firmware we are not
having right now. Maybe we have to dump additional memory regions.
That's what they did in the Cisco VoIP hack. They tried to find a memcpy routine and use it.
If you don't find a memcpy routine maybe you can implement your own. Why not?
You could dump code from other memory regions to output buffers.
If you have space in an external memory device, why not program it to the device
and read it from the device. It can be very interesting to gather more device internal information.
For example doing a RAM dump, because during static analysis, you always wonder
what may be in RAM at this and that address. Now as we have modified the firmware,
we can inject it back to the device. For example using the original updater.
It might contain the next checksum check, who knows.
We can try to re-program it to the external memory device if available, or to the processor.
This might be done using a serial interface, either JTAG or proprietary.
That's it. Thank you very much.
*applause*
Angel: If you have any questions, please line up at the room microphones, there are four here.
Are there any questions? Microphone 1 please;
Question: Not a question, but a tip. If you need some binary dump or some left over files on windows,
you can deny the delete right, so the install or updater program is unable to delete its tempfiles.
So they are left over after reprogramming the device.
A: Do you have tip what to use in that case?
Q: Sorry?
A: Do you have a tip, is there a special tool?
Q: It's not necessary, windows has the function already built in.
A: OK.
Q:And I don't know the word.
A: OK
Q: But you are able to revoke rights completely from a directory, there's a special right for deleting.
A: OK, thank you.
Angel: Are there any more questions?
Angel: Doesn't look like it, please give a warm round of applause to our speaker Stephan Widmann.
*applause* A: On [microphone 2]
Angel: There is one more question... No?
A: OK
Angel: If you're leaving please do take your...
subtitles created by c3subtitles.de