1 00:00:00,399 --> 00:00:09,269 ♪ (preroll music) ♪ 2 00:00:09,269 --> 00:00:15,829 Angel: After half a year, Volkswagen committed to tweaks to their emission readings. 3 00:00:15,829 --> 00:00:19,939 Those two boys, Daniel Lange and Felix Domke here on my left, 4 00:00:19,939 --> 00:00:22,170 will share some insights with us. 5 00:00:22,170 --> 00:00:24,330 Daniel will not only focus on the ECUs, 6 00:00:24,330 --> 00:00:26,759 which is the acronym for the Electronic Control Unit, 7 00:00:26,759 --> 00:00:29,599 and I think we're seeing one over here already, 8 00:00:29,599 --> 00:00:34,370 whereby Felix will show us some tricks to extract and tweak the firmware. 9 00:00:34,370 --> 00:00:37,750 On both sides we will see how many people have been involved in the entire process 10 00:00:37,750 --> 00:00:42,460 and we would get an idea what everything is involved in there. 11 00:00:42,460 --> 00:00:47,860 So, you applause and I'm gonna take over the Bildschirm. 12 00:00:47,860 --> 00:00:51,440 Good luck! 13 00:00:53,040 --> 00:00:56,810 Felix: Alright. Hello? Okay. 14 00:00:56,810 --> 00:01:01,210 Hey, so, I'm Felix Domke. 15 00:01:01,210 --> 00:01:04,220 Do we see the video output yet? No. 16 00:01:04,220 --> 00:01:05,740 Anyway, I'm Felix Domke. 17 00:01:05,740 --> 00:01:08,470 I'm here on my own because I was personally interested 18 00:01:08,470 --> 00:01:12,810 in how Volkswagen is cheating on their emission control. 19 00:01:12,810 --> 00:01:17,450 And maybe we get video at some point. 20 00:01:18,890 --> 00:01:21,150 I want to stress that I was self-funded. 21 00:01:21,150 --> 00:01:26,380 I did this with my own money because I was personally interested in this. 22 00:01:26,380 --> 00:01:30,280 So I did not do this on behalf of anyone else. 23 00:01:30,280 --> 00:01:34,869 Daniel: Let's start the Keynote again and see whether that one works better then. 24 00:01:34,869 --> 00:01:37,720 F: I am sure we figure this out. 25 00:01:37,720 --> 00:01:40,850 D: Oh, it worked before? 26 00:01:42,000 --> 00:01:47,140 Yes, that's because one of us two wanted to use a Mac. 27 00:01:47,140 --> 00:01:48,880 *audience laughs* 28 00:01:48,880 --> 00:01:52,690 F: But I wanted to use Keynote, I don't care which operating system. 29 00:01:52,690 --> 00:01:54,080 D: This one works. 30 00:01:54,080 --> 00:01:58,010 F: Anyway. So I will now hand off to Daniel which will give the first part of this talk 31 00:01:58,010 --> 00:02:01,020 and after that I will give the second part of this talk. 32 00:02:01,020 --> 00:02:04,290 D: Okay, thanks Felix. My name is Daniel. 33 00:02:04,290 --> 00:02:09,559 I used to work for a big Bavarian auto manufacturer which is not Audi... 34 00:02:09,559 --> 00:02:12,629 *audience laughing* 35 00:02:12,629 --> 00:02:14,189 ...for 14 years. 36 00:02:14,189 --> 00:02:19,129 I've been running the IT strategy, I've been doing IT architecture. 37 00:02:19,129 --> 00:02:20,909 And most relevant to this talk, 38 00:02:20,909 --> 00:02:26,200 I've been responsible for the process chain electronics and electric. 39 00:02:26,200 --> 00:02:30,680 I've done the rollout of Connected Drive in China. 40 00:02:30,680 --> 00:02:35,180 So I kind of have quite deep insight into how the automotive industry works 41 00:02:35,180 --> 00:02:37,219 and I'd like to share a bit with you. 42 00:02:37,219 --> 00:02:41,430 I'm an Engineer by training, I guess many of you are. 43 00:02:41,430 --> 00:02:45,269 And I want to share how the engineers think 44 00:02:45,269 --> 00:02:48,799 inside such a big corporation like Volkswagen, 45 00:02:48,799 --> 00:02:52,560 and what pressures, what boundary conditions they are working on. 46 00:02:52,560 --> 00:02:55,900 I have my own company now which makes my life a bit easier 47 00:02:55,900 --> 00:02:59,099 than Felix's, as you see in the legal disclaimer. 48 00:02:59,099 --> 00:03:04,400 Those are folks from the UK. 49 00:03:04,400 --> 00:03:10,150 They're called "Brandalism", I hope you notice the "McDonald's"-M at the end. 50 00:03:10,150 --> 00:03:15,450 Those are folks who started a few years ago to reclaim the public space. 51 00:03:15,450 --> 00:03:18,709 They were just annoyed by all of that advertising. 52 00:03:18,709 --> 00:03:24,049 And when the Paris negotiations for the eco treatment came, 53 00:03:24,049 --> 00:03:27,790 they just felt a big invitation to use the opportunity 54 00:03:27,790 --> 00:03:30,650 that Volkswagen has created for all of us 55 00:03:30,650 --> 00:03:34,639 and make advertising in their style, but perhaps not in the message 56 00:03:34,639 --> 00:03:36,879 they would usually have conveyed. 57 00:03:36,879 --> 00:03:40,749 I'm a strategist. 58 00:03:40,749 --> 00:03:46,939 So what is the thing that defines how the automotive indusry works today? 59 00:03:46,939 --> 00:03:50,709 We are in a saturated market. 60 00:03:50,709 --> 00:03:56,579 In the developed countries, so everywhere in Europe, in the North Americas, 61 00:03:56,579 --> 00:03:59,639 everybody has a car that wants one. Some have two. 62 00:03:59,639 --> 00:04:01,870 So when you want to sell another car 63 00:04:01,870 --> 00:04:07,049 you're basically talking about replacing an existing car with another one. 64 00:04:07,049 --> 00:04:09,779 The only growth you have is in the BRIC states. 65 00:04:09,779 --> 00:04:12,969 BRIC is: Brazil, Russia, India, and China. 66 00:04:12,969 --> 00:04:15,989 And, here, especially in China. 67 00:04:15,989 --> 00:04:19,978 You have a big overcapacity. There's just too many automotive manufacturers 68 00:04:19,978 --> 00:04:22,419 and there's too many plants they have. 69 00:04:22,419 --> 00:04:26,690 So all of them basically struggle to get the loads on the plants, 70 00:04:26,690 --> 00:04:31,729 to produce enough cars and to have those cars sold at some point in time. 71 00:04:31,729 --> 00:04:35,050 Because the queueing in between production and sales 72 00:04:35,050 --> 00:04:41,849 is actually the big parking spaces you see in Bremerhaven or so 73 00:04:41,849 --> 00:04:45,979 where there's ten thousands, in some countries even hundreds of thousands of cars 74 00:04:45,979 --> 00:04:50,800 basically stored in between production and sales. 75 00:04:50,800 --> 00:04:53,629 Ten years ago, fifteen years ago, that didn't exist. 76 00:04:53,629 --> 00:04:58,419 The cars were basically sold off the factory. 77 00:04:58,419 --> 00:05:02,770 But people have been moving away very, very slowly from cars. 78 00:05:02,770 --> 00:05:04,539 They have, as I said, a saturated market. 79 00:05:04,539 --> 00:05:08,819 It's just not that easy to sell a car anymore. 80 00:05:08,819 --> 00:05:12,259 That is because of social shifts. 81 00:05:12,259 --> 00:05:15,620 When I was young, there was "The Dukes of Hazzard", 82 00:05:15,620 --> 00:05:20,289 there was "General Lee", this car that basically is the star of the show. 83 00:05:20,289 --> 00:05:24,500 There was "Knight Rider" and nobody watched it for David Hasselhoff, not even the girls. 84 00:05:24,500 --> 00:05:27,870 They watched it for KITT. 85 00:05:27,870 --> 00:05:30,720 When I was young, I wanted to own a car. 86 00:05:30,720 --> 00:05:33,810 I wanted to have KITT, possibly. 87 00:05:33,810 --> 00:05:37,430 And when I grew old enough, I found out I can get a car that looks like KITT 88 00:05:37,430 --> 00:05:39,710 but, you know, all the fun stuff is not in there, 89 00:05:39,710 --> 00:05:43,840 so I turned to computers. 90 00:05:43,840 --> 00:05:48,710 The next thing is organization, megacities. 91 00:05:48,710 --> 00:05:51,259 We live in very condensed spaces in those cities. 92 00:05:51,259 --> 00:05:56,460 If you talk about a place like Beijing where there's like 21 million people 93 00:05:56,460 --> 00:06:01,669 in an area that is one city, where there's nothing big inbetween, 94 00:06:01,669 --> 00:06:04,259 there's no river, there's no forest, 95 00:06:04,259 --> 00:06:06,389 it's just like one city. 96 00:06:06,389 --> 00:06:09,990 If you go to Tokyo, Yokohama, you can drive on the motorway 97 00:06:09,990 --> 00:06:14,879 for nearly three hours when you enter the city before you leave the city. 98 00:06:14,879 --> 00:06:18,229 And you're driving on the motorway, you're driving on an elevated road 99 00:06:18,229 --> 00:06:21,620 for which you paid toll so you actually can drive. 100 00:06:21,620 --> 00:06:25,099 But it's three hours before you leave the city again. 101 00:06:25,099 --> 00:06:28,050 And in these cities owning a car and operating a car 102 00:06:28,050 --> 00:06:31,710 is about the worst thing you can do. You just don't want to do that. 103 00:06:31,710 --> 00:06:41,039 The average speed of a car in Beijing these days is 12 km/h. 104 00:06:41,039 --> 00:06:43,819 If you're a good runner you can beat that. 105 00:06:43,819 --> 00:06:49,180 And incidentally this is exactly the speed that a horse carriage makes. 106 00:06:49,180 --> 00:06:55,889 *audience laughs and applauds* 107 00:06:55,889 --> 00:07:01,039 We have managed to undo all of the innovation of the last 200 years, 108 00:07:01,039 --> 00:07:05,729 it's just the interior is a little bit more comfortable. 109 00:07:05,729 --> 00:07:10,189 The actual getting from A to B is the same as with a horse carriage these days. 110 00:07:10,189 --> 00:07:12,330 And then there's technology shifts. 111 00:07:12,330 --> 00:07:17,879 The problem is, there are big things, big visions that everybody follows, 112 00:07:17,879 --> 00:07:20,919 like electric mobility. Electric mobility means: 113 00:07:20,919 --> 00:07:24,449 You buy a car that's one and a half times the price of your standard car, 114 00:07:24,449 --> 00:07:28,870 you lug around 300 kg of batteries for no apparent reason to do so, 115 00:07:28,870 --> 00:07:31,789 and you now need to install something in your garage — 116 00:07:31,789 --> 00:07:34,780 which you most probably don't have, look at megacities — 117 00:07:34,780 --> 00:07:38,740 to be able to recharge the car because it only goes a hundred miles. 118 00:07:38,740 --> 00:07:44,960 So it's currently not a very compelling thing to sell to the end customer. 119 00:07:44,960 --> 00:07:49,430 There's self-driving cars, which is kind of a great, big vision. 120 00:07:49,430 --> 00:07:51,520 But I would really call that a "vision". 121 00:07:51,520 --> 00:07:56,169 A "vision" is something that's not being implemented in my lifetime. 122 00:07:56,169 --> 00:07:57,569 And then there's downsizing. 123 00:07:57,569 --> 00:07:59,689 Downsizing means ... 124 00:07:59,689 --> 00:08:01,699 Everybody wanted to have the biggest engine, 125 00:08:01,699 --> 00:08:04,770 everybody wanted to have the biggest car, let's say, 10 years ago. 126 00:08:04,770 --> 00:08:08,580 You wanted to have that six cylinder that was giving you status. 127 00:08:08,580 --> 00:08:12,289 But now the automotive industry has an overall cap 128 00:08:12,289 --> 00:08:17,469 on how much emissions the average new car fleet my have. 129 00:08:17,469 --> 00:08:22,009 And they can only reach that if they manage to sell smaller engines to you. 130 00:08:22,009 --> 00:08:24,349 Because for everybody who buys a really big engine 131 00:08:24,349 --> 00:08:26,999 that will never ever make that emission cap, 132 00:08:26,999 --> 00:08:30,240 they need somebody whom they've sold a small car to — 133 00:08:30,240 --> 00:08:34,909 preferrably an electric car, because they even have statistical advantages 134 00:08:34,909 --> 00:08:37,220 to make them a bit more attractive — 135 00:08:37,220 --> 00:08:38,820 to set that off. 136 00:08:38,820 --> 00:08:43,230 So very literally the poor guy with the small car needs to exist 137 00:08:43,230 --> 00:08:47,850 for the rich guy that drives the eight cylinder and doesn't give a shit. 138 00:08:50,700 --> 00:08:57,940 Strategy-wise, there's only two things that an automotive car company is driven in. 139 00:08:57,940 --> 00:09:00,980 And that's really everything there is. 140 00:09:00,980 --> 00:09:03,340 There is "reach a target ROCE". 141 00:09:03,340 --> 00:09:07,260 ROCE is: Return on Capital Employed. That is just two numbers: 142 00:09:07,260 --> 00:09:12,200 your EBIT, which is your Earnings Before Income Tax, 143 00:09:12,200 --> 00:09:17,230 and the amount of money you have in your company, the employed capital, 144 00:09:17,230 --> 00:09:19,160 which you got from people that lent it to you 145 00:09:19,160 --> 00:09:22,660 or from your stakeholders, from your investors. 146 00:09:22,660 --> 00:09:25,520 And that is what the company is measured against. 147 00:09:25,520 --> 00:09:27,900 Every automotive company basically runs like this. 148 00:09:27,900 --> 00:09:30,870 Just this one figure, it's a percentage like "30%." 149 00:09:30,870 --> 00:09:39,330 "30%" means: On the money you have you made a 30% return during that year. 150 00:09:39,330 --> 00:09:44,750 The downside of measuring in ROCE is that everytime you use that Euro or Dollar 151 00:09:44,750 --> 00:09:48,520 it counts again because the money works for you. 152 00:09:48,520 --> 00:09:50,010 That means what you're looking at 153 00:09:50,010 --> 00:09:55,480 is a company that gradually moves from a very industrial type of application 154 00:09:55,480 --> 00:09:57,370 to something that tries to move faster, 155 00:09:57,370 --> 00:10:03,180 that tries to be quick and regain money faster. 156 00:10:03,180 --> 00:10:05,100 And then there's "outperform the competition." 157 00:10:05,100 --> 00:10:08,200 You have to understand the situation 158 00:10:08,200 --> 00:10:10,790 that there's a good dozen companies 159 00:10:10,790 --> 00:10:14,420 and everybody has the same strategic position: 160 00:10:14,420 --> 00:10:17,240 "We will outperform the competition." 161 00:10:17,240 --> 00:10:20,140 So statistically, you will know that half of them are going to fail 162 00:10:20,140 --> 00:10:25,460 because that won't happen, right? Somebody has to be the lower half. 163 00:10:25,460 --> 00:10:29,220 But the only thing I have seen in about five or six companies 164 00:10:29,220 --> 00:10:33,180 where I know the strategy in detail, is: the sequence. 165 00:10:33,180 --> 00:10:36,680 Is the first or is the latter the more important one? 166 00:10:36,680 --> 00:10:38,710 And sometimes that depends on markets. 167 00:10:38,710 --> 00:10:41,890 There's this new emerging market and you want to outperform the competition, 168 00:10:41,890 --> 00:10:43,620 you want to grow more. 169 00:10:43,620 --> 00:10:48,180 And then there's this laggard market somewhere in the European Union 170 00:10:48,180 --> 00:10:51,870 where you just look at the money, you know, how much money are we making on this. 171 00:10:51,870 --> 00:10:56,810 But that's all, that is how an engineer is basically steered, that is the strategy. 172 00:10:56,810 --> 00:11:03,530 And that means when you break that down through the levels of hierarchy, what is counting is: 173 00:11:03,530 --> 00:11:07,330 How much money do you need to make this? 174 00:11:07,330 --> 00:11:09,720 How much money are you gonna make on this? 175 00:11:09,720 --> 00:11:14,080 Those two divided will be contributing to the ROCE. 176 00:11:14,080 --> 00:11:20,320 And do you deliver anything that can help us outperform the competition? 177 00:11:20,320 --> 00:11:24,380 You notice that there is a lack, which is, you know: What does the customer want? 178 00:11:24,380 --> 00:11:27,790 Or: What is good for associates? Or something like that. 179 00:11:27,790 --> 00:11:30,720 Just in case you hadn't noticed before. 180 00:11:30,720 --> 00:11:37,460 Okay, I'd like to do a bit of a quiz with you before you all fall asleep after lunch. 181 00:11:37,460 --> 00:11:39,270 Eleven million. 182 00:11:39,270 --> 00:11:42,800 "Eleven million" in the context of the exhaust emission scandal. 183 00:11:42,800 --> 00:11:44,320 What is that number? 184 00:11:44,320 --> 00:11:46,890 Audience: Cars affected! 185 00:11:46,890 --> 00:11:48,830 Correct! Cars affected. 186 00:11:48,830 --> 00:11:51,960 Eleven million is actually the Volkswagen cars 187 00:11:51,960 --> 00:11:54,910 which need to be recalled world-wide 188 00:11:54,910 --> 00:12:00,050 to get this little filter thing fixed and their software updated 189 00:12:00,050 --> 00:12:04,880 to meet the emission targets which they had been produced against. 190 00:12:04,880 --> 00:12:07,580 1500 ...? 191 00:12:09,810 --> 00:12:12,380 A: Number of engineers! 192 00:12:12,380 --> 00:12:14,470 Number of engineers? No, not correct. 193 00:12:14,470 --> 00:12:20,420 Number of engineers would be above 10000 for a car in Volkswagen Group. Sorry? 194 00:12:20,420 --> 00:12:22,880 A: Cost for fixing it per car? 195 00:12:22,880 --> 00:12:26,750 Cost for fixing it per car? No, that's maximum 600, we're gonna see later. 196 00:12:26,750 --> 00:12:28,580 *unintelligible suggestion from audience* 197 00:12:28,580 --> 00:12:32,100 No? Well that was too difficult then, and that was a bit intentional. 198 00:12:32,100 --> 00:12:37,080 That's the amount of hard disks they collected from the associates. 199 00:12:37,080 --> 00:12:42,420 *audience laughs and applauds* 200 00:12:42,420 --> 00:12:48,190 Now the thing is, we've had in the press that there is maximum 13 managers 201 00:12:48,190 --> 00:12:53,740 which are responsible for this emission scandal within Volkswagen. 202 00:12:53,740 --> 00:12:59,780 But then they collect 1500 hard disks and USB sticks from 380 associates, 203 00:12:59,780 --> 00:13:03,670 and that number is a month old because they haven't reported newer numbers. 204 00:13:03,670 --> 00:13:07,680 So something is mismatching there, right? Something is mismatching there. 205 00:13:07,680 --> 00:13:09,980 So the first number we have is for how many associates 206 00:13:09,980 --> 00:13:13,190 are actually somehow affected by this is 380. 207 00:13:13,190 --> 00:13:17,780 Because you come to work somewhere in Wolfsburg I think, right? 208 00:13:17,780 --> 00:13:21,230 And then there's this nice chap coming up and telling you, 209 00:13:21,230 --> 00:13:24,170 "Uhm, actually we took the hard disk off your PC, 210 00:13:24,170 --> 00:13:26,880 you're gonna get a new one from IT, we guess tomorrow, 211 00:13:26,880 --> 00:13:31,920 they're a bit behind with, you know ..." 212 00:13:31,920 --> 00:13:35,190 6.7 billion ...? 213 00:13:35,190 --> 00:13:37,260 Just shout! 214 00:13:37,260 --> 00:13:39,400 *unintelligible suggestions from audience* 215 00:13:39,400 --> 00:13:42,640 Fine? No that will be less, much less. 216 00:13:42,640 --> 00:13:44,050 *unintelligible suggestions from audience* 217 00:13:44,050 --> 00:13:46,880 Yes, you're getting close. It's the money they put back, 218 00:13:46,880 --> 00:13:52,220 they set aside to actually pay for the recall and the legal fees. 219 00:13:52,220 --> 00:13:57,330 Now if you divide that by 11 million you get about €600 per car. 220 00:13:57,330 --> 00:14:01,760 So it's not that much money per car. 221 00:14:01,760 --> 00:14:08,040 In Europe, the plan is basically that you go to the dealer and get a software update. 222 00:14:08,040 --> 00:14:14,980 In the States, people already got $1000 in cash and in coupons 223 00:14:14,980 --> 00:14:17,260 as a goodwill measure. 224 00:14:17,260 --> 00:14:22,770 So something I learnt from Martin Haase here going to the CCC Congress all the time 225 00:14:22,770 --> 00:14:26,150 is that we need to read text really well. 226 00:14:26,150 --> 00:14:30,350 So the upper one is the original in German, the lower one is my English translation. 227 00:14:30,350 --> 00:14:34,870 The English translation is as accurate as possible, so it's not good English. 228 00:14:34,870 --> 00:14:40,070 Please excuse that, it is so you get the gist in case you can only read the English. 229 00:14:40,070 --> 00:14:44,400 So that is Mr. Pötsch, he's the president of the Volkswagen supervisory board. 230 00:14:44,400 --> 00:14:50,170 He is the poor guy that now has to sort it all out. 231 00:14:50,170 --> 00:14:55,290 He used to be the CFO. We're gonna see why that is important a little bit later. 232 00:14:55,290 --> 00:14:58,740 And he has made this analysis: 233 00:14:58,740 --> 00:15:03,870 It was "individual misbehaviour", so it's not an organizational problem, 234 00:15:03,870 --> 00:15:06,630 it's "weaknesses in particular processes", 235 00:15:06,630 --> 00:15:10,250 and it's "the attitude in particular sub-partitions" ... 236 00:15:10,250 --> 00:15:14,060 "Teilbereiche des Unternehmens", it's impossible to translate in English, 237 00:15:14,060 --> 00:15:19,850 it's actually impossible in German, but, you know, the legal team came up with that. 238 00:15:19,850 --> 00:15:24,620 So the "attitude in particular sub-partitions of the company to tolerate rule violations." 239 00:15:24,620 --> 00:15:28,590 Now, if we go through this very quickly: It's not a rule violation, 240 00:15:28,590 --> 00:15:32,050 you violated the fucking law. 241 00:15:32,050 --> 00:15:37,160 The other thing is, if you have particular processes, you have particular associates, 242 00:15:37,160 --> 00:15:40,010 and you have particular sub-partitions of the company, 243 00:15:40,010 --> 00:15:43,560 That tells you something, right? That just tells you something. 244 00:15:43,560 --> 00:15:46,610 This was probably two days' work of somebody in the legal team, 245 00:15:46,610 --> 00:15:51,030 and I guess you noticed, right? I guess you notice. 246 00:15:51,030 --> 00:15:55,680 "Legal team" is probably these people. 247 00:15:55,680 --> 00:16:02,040 Jones Day is a big American lawyer company 248 00:16:02,040 --> 00:16:05,320 and they've asked them to help with sorting out this. 249 00:16:05,320 --> 00:16:11,530 Now the funny thing is, there's public prosecutors all over the planet interested in Volkswagen 250 00:16:11,530 --> 00:16:13,980 but Volkswagen thinks it's not really clever 251 00:16:13,980 --> 00:16:17,320 to have those people come in and find all the info, 252 00:16:17,320 --> 00:16:22,020 it's better to have Jones Day, their own kind of bought-in legal team, 253 00:16:22,020 --> 00:16:24,690 ask the associates first. 254 00:16:24,690 --> 00:16:31,040 Now the problem is, whenever the let's say German prosecutors wake up and go in there 255 00:16:31,040 --> 00:16:36,120 and say, like, "We would like to see what has happened, 256 00:16:36,120 --> 00:16:39,530 so please hand over the material, please hand over the hard disks," 257 00:16:39,530 --> 00:16:44,630 they would get a very, very nice reception, be greeted with coffee and shown a room 258 00:16:44,630 --> 00:16:47,100 where all of the hard disks and everything is stored, 259 00:16:47,100 --> 00:16:51,990 "We collected it for you." 260 00:16:51,990 --> 00:16:55,860 I have no idea whether they gonna show everything to them 261 00:16:55,860 --> 00:17:00,890 I have no idea whether there may be some material lost in between. 262 00:17:00,890 --> 00:17:04,680 We've heard from Anna earlier in Germany it seems to be 263 00:17:04,680 --> 00:17:08,339 that things hit the shredder and hard disks get lost and everything. 264 00:17:08,339 --> 00:17:13,010 So if it works like that in the government I have no idea how it works in companies. 265 00:17:13,010 --> 00:17:17,540 But if I was on the prosecutors I'd probably see that I speed up a little 266 00:17:17,540 --> 00:17:20,540 because otherwise you'll get all pre-prepared material. 267 00:17:20,540 --> 00:17:23,010 And because Jones Day can't do all of that — 268 00:17:23,010 --> 00:17:26,310 you have to interview all of those people, and you have to look through the hard disks — 269 00:17:26,310 --> 00:17:29,450 they asked Deloitte to come in and help them. 270 00:17:29,450 --> 00:17:32,810 Now Deloitte are a very good company, they have very, very good forensic teams, 271 00:17:32,810 --> 00:17:36,020 so that's a very good choice. But the important thing here is: 272 00:17:36,020 --> 00:17:41,160 Out of the four big consulting companies that do finance analysis and stuff 273 00:17:41,160 --> 00:17:45,420 those are the only Americans. The others are headquartered somewhere else. 274 00:17:45,420 --> 00:17:52,080 So what it tells you here — American legal teams, American auditors — 275 00:17:52,080 --> 00:17:55,940 that's where Volkswagen looks. Volkswagen is actually afraid of America. 276 00:17:55,940 --> 00:18:02,510 They are not that afraid of Europe or some other country in some other continent. 277 00:18:05,110 --> 00:18:08,990 Now, let's talk text a bit again. 278 00:18:10,910 --> 00:18:16,750 "We have no findings on the involvement of the supervisory board 279 00:18:16,750 --> 00:18:19,630 or the board of management presented." 280 00:18:19,630 --> 00:18:24,590 Now, again, "no findings", okay, "presented", right? 281 00:18:24,590 --> 00:18:28,200 It's not "we don't have any findings" or "there is nothing", 282 00:18:28,200 --> 00:18:30,810 it says "we have no findings presented." 283 00:18:30,810 --> 00:18:34,040 And the other thing is "involvement", that's an odd term. 284 00:18:34,040 --> 00:18:36,670 In German, "Involvierung", that's not even German, right? 285 00:18:36,670 --> 00:18:40,610 If you look it up, "Involvierung", nobody of you talks of "Involvierung" 286 00:18:40,610 --> 00:18:44,260 when you talk to your family or when you do something at work. 287 00:18:44,260 --> 00:18:49,750 The trick here is, the supervisory board has a reason for existing: supervision. 288 00:18:49,750 --> 00:18:54,480 *audience laughs and applauds* 289 00:18:54,480 --> 00:18:59,140 The board of management has a reason for existing and that is: decision. 290 00:18:59,140 --> 00:19:01,050 They are the deciding body. 291 00:19:01,050 --> 00:19:04,500 None of them are ever "involved", right? 292 00:19:04,500 --> 00:19:07,740 When you work on something in a big hierarchical company 293 00:19:07,740 --> 00:19:10,840 there is no "involvement" of your board member, 294 00:19:10,840 --> 00:19:14,380 there is no "involvement" of your supervisory board member. 295 00:19:14,380 --> 00:19:18,120 So per definition, they cannot have an involvement, right? 296 00:19:18,120 --> 00:19:21,290 If he wanted to be straight he would have said: 297 00:19:21,290 --> 00:19:27,250 "I, as a former board of management director and now as the head of the supervisory board, 298 00:19:27,250 --> 00:19:32,520 I guarantee there was no involvement of my or my colleagues in this. 299 00:19:32,520 --> 00:19:37,370 And if there was, I would pay back my salary, I will go to jail, I will ... whatever." 300 00:19:37,370 --> 00:19:40,320 Right, sacrifice a goat? 301 00:19:40,320 --> 00:19:42,970 But that would have been straight communication. 302 00:19:42,970 --> 00:19:47,250 But this is not straight communication, this is... bullshit. 303 00:19:48,430 --> 00:19:52,020 Okay, quiz time! 10 ...? 304 00:19:52,020 --> 00:19:59,070 You remember, this guy here told us there's no involvement in anything fishy, right? 305 00:19:59,070 --> 00:20:03,290 It's all those small engineers, all those bad, bad people down there. 306 00:20:03,290 --> 00:20:05,460 But they are gonna hunt 'em down, right? 307 00:20:05,460 --> 00:20:08,110 So there's no involvement with anything fishy here. 308 00:20:08,110 --> 00:20:10,530 So, in that context, what is "10"? 309 00:20:10,530 --> 00:20:12,140 A: Board members! 310 00:20:12,140 --> 00:20:14,190 10 board members? Close, they have a little more. 311 00:20:14,190 --> 00:20:15,990 A: Levels of hierarchy. 312 00:20:15,990 --> 00:20:20,120 Levels of hierarchy — quite good. It's, I think, eight or so, but you're quite close. 313 00:20:20,120 --> 00:20:29,120 No, it's actually the amount of planes that Volkswagen owns. 314 00:20:29,120 --> 00:20:30,930 All of them are jet planes. 315 00:20:30,930 --> 00:20:35,260 Because if you're a board member you have to, you know, fly in style. 316 00:20:35,260 --> 00:20:38,740 And because there's nothing ever fishy at Volkswagen 317 00:20:38,740 --> 00:20:43,670 it's run by Lion Air Services out of the Braunschweig airport. 318 00:20:43,670 --> 00:20:49,870 And obviously, Lion Air Servives is registered in Georgetown on the Cayman Islands. 319 00:20:49,870 --> 00:20:52,870 *applause and laughter* 320 00:20:52,870 --> 00:20:54,930 Nothing fishy ever in that company. 321 00:20:54,930 --> 00:20:58,190 Okay, let's get back to topic. I have about another ten minutes 322 00:20:58,190 --> 00:21:03,450 before I want to get Felix the chance to show you what he has done on the ECUs. 323 00:21:03,450 --> 00:21:09,940 So I need to get you up to speed about how all of this context here works. 324 00:21:09,940 --> 00:21:15,930 And this here is called the NEDC, it's the New European Driving Cycle. 325 00:21:15,930 --> 00:21:20,860 This is what your car is tested against for emissions. 326 00:21:20,860 --> 00:21:24,620 It works like that: You condition the vehicle a day before. 327 00:21:24,620 --> 00:21:27,340 Which means you really drive it hard on the Autobahn 328 00:21:27,340 --> 00:21:31,370 so the exhaust is really free and everything. 329 00:21:31,370 --> 00:21:37,490 And then you do these cycles here where you basically accelerate the vehicle, 330 00:21:37,490 --> 00:21:40,200 slow down, accelerate the vehicle, slow down, accelerate the vehicle, 331 00:21:40,200 --> 00:21:43,680 slow a bit down, slow a bit more down, and then you cycle again. 332 00:21:43,680 --> 00:21:47,090 And the last, cycle 5, is an optional one, depending on what you measure, 333 00:21:47,090 --> 00:21:49,390 that is actually going to the autobahn 334 00:21:49,390 --> 00:21:56,280 and you're going up to a top speed of 120 km/h for a very short period of time. 335 00:21:56,280 --> 00:22:04,850 The people that have detected the tweaked emissions 336 00:22:04,850 --> 00:22:11,230 in the VW Jetta and Passat they looked at, they have called this 337 00:22:11,230 --> 00:22:16,450 "a very light usage cycle," and they called it "unrealistic". 338 00:22:16,450 --> 00:22:21,260 Because basically nobody drives the car like this, it's a very artificial thing. 339 00:22:21,260 --> 00:22:23,210 And that is the problem for the engineer, right? 340 00:22:23,210 --> 00:22:27,630 The engineer looks at this and says, "Yeah, you know, it's a standard. 341 00:22:27,630 --> 00:22:30,010 It's something we do to measure against." 342 00:22:30,010 --> 00:22:32,930 But nobody drives like this. It's not realistic, right? 343 00:22:32,930 --> 00:22:37,680 So if you fake the data in this we're not actually faking something our customer uses 344 00:22:37,680 --> 00:22:41,800 because no customer drives like this, it's very artificial. 345 00:22:41,800 --> 00:22:46,200 And there's a very good report by ICCT which is, "Mind the Gap". 346 00:22:46,200 --> 00:22:49,250 Which is what you hear in London when you go into the Tube. 347 00:22:49,250 --> 00:22:55,059 And what they mean is the gap between what gets out when you measure emissions like this 348 00:22:55,059 --> 00:22:57,850 and what gets out when you actually drive the car. 349 00:22:57,850 --> 00:23:00,870 And that gap is widening year by year by year. 350 00:23:00,870 --> 00:23:05,740 Because engineers get better and better at optimizing for this cycle. 351 00:23:05,740 --> 00:23:10,770 The cars on the street? Phhh, they do get better as well, but less, right? 352 00:23:10,770 --> 00:23:14,140 That's why the gap widens. 353 00:23:14,140 --> 00:23:17,420 And trickery on those tests is very common. 354 00:23:17,420 --> 00:23:21,130 I'm sorry you can't probably read that in the stream 355 00:23:21,130 --> 00:23:23,800 and probably can't read that when you're back down there. 356 00:23:23,800 --> 00:23:27,010 But that's an original slide I had to take from Transport & Environment, 357 00:23:27,010 --> 00:23:29,180 from that report which I just named. 358 00:23:29,180 --> 00:23:32,660 And what it says there is what tricks people are doing 359 00:23:32,660 --> 00:23:35,320 to actually drive down the emissions. 360 00:23:35,320 --> 00:23:37,900 For example, they blow up the tyres 361 00:23:37,900 --> 00:23:41,770 by 3 bars more than you could actually use them on the road. 362 00:23:41,770 --> 00:23:45,600 Now when you do, the bottom of the tyre looks like this, right? 363 00:23:45,600 --> 00:23:48,340 So that means you only have a very, very small portion of the tyre 364 00:23:48,340 --> 00:23:52,540 that still touches the ground, so your resistance gets reduced. 365 00:23:52,540 --> 00:23:59,760 They put diesel into the oil beause diesel is lighter than the oil which you are using 366 00:23:59,760 --> 00:24:02,740 inside the vehicle, so friction gets reduced. 367 00:24:02,740 --> 00:24:07,320 They take off the mirror, the side mirror on the passenger side 368 00:24:07,320 --> 00:24:10,290 because that is not legally required to be existing, 369 00:24:10,290 --> 00:24:14,170 so, you know, it's resistance, so get away with it. 370 00:24:14,170 --> 00:24:17,790 They tape close all of the openings of the vehicle 371 00:24:17,790 --> 00:24:19,980 because obviously when the wind goes over it 372 00:24:19,980 --> 00:24:23,320 it goes much smoother once you have everything taped. 373 00:24:23,320 --> 00:24:30,220 Now all of these things are either okay or they are kind of borderline grey area. 374 00:24:30,220 --> 00:24:33,179 And they do this. This is how actually emissions are tested. 375 00:24:33,179 --> 00:24:36,170 So this is why an engineer, when he looks at this, says, 376 00:24:36,170 --> 00:24:40,160 "Yeah, it's an optimization problem. They want me to get a low number 377 00:24:40,160 --> 00:24:45,780 and I have pretty clever ideas, which involve diesel and sticky tape and everything, 378 00:24:45,780 --> 00:24:47,460 to reduce the number." 379 00:24:47,460 --> 00:24:50,460 *sighs* 380 00:24:50,460 --> 00:24:53,140 The results are this. 381 00:24:53,140 --> 00:24:58,670 That's from a 2012 report from — a 2013 report, I'm sorry — from ADAC, 382 00:24:58,670 --> 00:25:02,860 the German MRT company. 383 00:25:02,860 --> 00:25:07,370 And what you see, the lighter blue ones are actually the emissions 384 00:25:07,370 --> 00:25:11,679 which the car produces in this cycle. 385 00:25:11,679 --> 00:25:14,170 The darker blue ones are the ones which are produced 386 00:25:14,170 --> 00:25:17,490 when you just go on the motorway and drive them. 387 00:25:17,490 --> 00:25:19,140 And you see that there is a discrepancy 388 00:25:19,140 --> 00:25:26,960 which is ten times, twenty times, thirty times what is the measured data. 389 00:25:26,960 --> 00:25:31,610 So what you need to understand is that even in the past nobody ever thought, 390 00:25:31,610 --> 00:25:35,980 nobody in the industry ever thought that the data which was measured 391 00:25:35,980 --> 00:25:40,160 had any real connection with reality, right? 392 00:25:40,160 --> 00:25:43,510 The only connection was, you knew that what you're measuring 393 00:25:43,510 --> 00:25:48,090 within the duty cycle NEDC is definitely less 394 00:25:48,090 --> 00:25:51,080 than what you would ever see in any realtime use. 395 00:25:51,080 --> 00:25:54,410 But that's it, that's it. That's no secret, right? 396 00:25:54,410 --> 00:25:58,470 It's something that has been out there for years. 397 00:25:59,630 --> 00:26:04,690 Now the folks at Deutsche Umwelthilfe, which are actually people that helped 398 00:26:04,690 --> 00:26:09,130 find out what Volkswagen did, they wanted to see that others do it as well. 399 00:26:09,130 --> 00:26:12,460 And because I wanted to give you as much information as possible 400 00:26:12,460 --> 00:26:17,910 we are going to look at this product here now, which is not a Volkswagen as you may see. 401 00:26:17,910 --> 00:26:21,640 And when you measure this car it actually looks like this. 402 00:26:21,640 --> 00:26:26,809 So that means when the car is thinking it is running an NEDC — 403 00:26:26,809 --> 00:26:30,880 because it is conditioned to do so, it is the right temperature, 404 00:26:30,880 --> 00:26:35,000 it is the right setup — it actually delivers the blue bars. 405 00:26:35,000 --> 00:26:39,760 And if you run it because you just run it and you don't do the conditioning 406 00:26:39,760 --> 00:26:42,070 it delivers the grey bars. 407 00:26:42,070 --> 00:26:45,610 Now there's many things you can say about how they measured this 408 00:26:45,610 --> 00:26:52,020 because, obviously, this is not science to the best level of accuracy. 409 00:26:52,020 --> 00:26:54,980 But you do see a pattern here, and you do see the pattern 410 00:26:54,980 --> 00:26:59,610 of the 30-, 35-fold emissions. And that is what you always see 411 00:26:59,610 --> 00:27:03,170 because this is what an engine like the one in this car — 412 00:27:03,170 --> 00:27:06,370 a 1.6 l diesel engine if I remember correctly — 413 00:27:06,370 --> 00:27:10,600 actually does when it's just operated normally. 414 00:27:10,600 --> 00:27:16,800 And the lower ones are the ones which you get when the engineers did all the good tweaking. 415 00:27:18,420 --> 00:27:23,660 Now why has all of this ... Oh sorry, so this is just one test, right? 416 00:27:23,660 --> 00:27:28,540 And you see that this test, when the vehicle is cold, 417 00:27:28,540 --> 00:27:34,799 you get fresh air with a nice rose smell out of the exhaust. 418 00:27:34,799 --> 00:27:38,660 And when the vehicle is operated normally you basically get what you expect, 419 00:27:38,660 --> 00:27:42,309 you get the combustion products out of burning diesel. 420 00:27:42,309 --> 00:27:44,929 Now why is all of this now a problem? 421 00:27:44,929 --> 00:27:48,110 This is now a problem because of the American legal system. 422 00:27:48,110 --> 00:27:50,380 The American legal system is very, very different 423 00:27:50,380 --> 00:27:55,370 from what people in the European Union are used to. 424 00:27:55,370 --> 00:27:59,510 In America, there are two things which are a bit strange perhaps 425 00:27:59,510 --> 00:28:03,570 to somebody who's accustomed with a German legal system. 426 00:28:03,570 --> 00:28:05,580 The first thing is, there's jurys. 427 00:28:05,580 --> 00:28:10,490 So there's common people that actually decide about what's right or what's wrong. 428 00:28:10,490 --> 00:28:14,260 And that means, what they award as compensation 429 00:28:14,260 --> 00:28:20,390 to people that have had a disadvantage are often astronomic figures. 430 00:28:20,390 --> 00:28:23,690 Now these figures are sometimes reduced again by the judges, 431 00:28:23,690 --> 00:28:30,080 but it's not uncommon that if something hurt you or you got into an accident 432 00:28:30,080 --> 00:28:32,620 you're awarded million dollar sums. 433 00:28:32,620 --> 00:28:37,150 In Germany, if somebody shoots your eye out, you may be getting €100,000. 434 00:28:37,150 --> 00:28:39,790 So there's a huge discrepancy there. 435 00:28:39,790 --> 00:28:43,290 And the other thing is, in America there are punitive damages. 436 00:28:43,290 --> 00:28:47,830 "Punitive damages" means: You did something wrong, you did it on purpose, 437 00:28:47,830 --> 00:28:50,420 and you're punished for it. 438 00:28:50,420 --> 00:28:54,200 In Europe, a company basically is, you did something wrong 439 00:28:54,200 --> 00:28:58,580 so now you have to compensate the disadvantage somebody else had. 440 00:28:58,580 --> 00:29:03,400 So to a certain extent, a company that doesn't try to trick 441 00:29:03,400 --> 00:29:09,679 actually kind of loses an opportunity because if they are not detected to be tricking 442 00:29:09,679 --> 00:29:11,320 they have just saved money. 443 00:29:11,320 --> 00:29:15,360 There's no punitive element, there's no "You will go to jail for this." 444 00:29:15,360 --> 00:29:19,950 At least in this context of environmental regulation. 445 00:29:19,950 --> 00:29:23,799 Now in case you couldn't read that, that's actually a sign I took in california. 446 00:29:23,799 --> 00:29:27,850 You go into a store and it tells you that basically everything you see there 447 00:29:27,850 --> 00:29:33,780 and touch there is giving you cancer and your unborn children will be damaged. 448 00:29:33,780 --> 00:29:36,240 This is what it says there: Belts, shoes, jewellery, handbags, 449 00:29:36,240 --> 00:29:38,030 all products with metal, and everything 450 00:29:38,030 --> 00:29:41,929 causes cancer, birth defects, and other reproducive damages. 451 00:29:41,929 --> 00:29:43,809 So this is America, right? 452 00:29:43,809 --> 00:29:51,030 Their view of protecting the consumer is completely different from Europe. 453 00:29:51,030 --> 00:29:54,670 And this is why Volkswagen goes and says, "We will show good faith. 454 00:29:54,670 --> 00:30:00,130 We will give you, American Volkswagen owner, a thousand dollars 455 00:30:00,130 --> 00:30:05,340 because we just wanna make sure that you at least know we care." 456 00:30:05,340 --> 00:30:07,790 It's important that you care because the jury will say, 457 00:30:07,790 --> 00:30:11,160 "Well at least they awarded $1000, maybe a little too little, 458 00:30:11,160 --> 00:30:13,100 but at least they did something." 459 00:30:13,100 --> 00:30:18,210 The jury would say that. A professional judge in Germany would say, "Pshh, why?" 460 00:30:18,210 --> 00:30:22,880 So this is why as a European customer you actually go to the dealership, 461 00:30:22,880 --> 00:30:25,490 and if that guy is really nice you may be getting a coffee 462 00:30:25,490 --> 00:30:28,660 while you wait the hour that he flashes your car. 463 00:30:28,660 --> 00:30:34,160 So that's the only thing you're currently supposedly getting in Europe. 464 00:30:34,160 --> 00:30:38,799 Okay, now the problem is: What they did hurts. 465 00:30:38,799 --> 00:30:42,230 And it hurts because, if you do the statistics ... 466 00:30:42,230 --> 00:30:49,820 Very nice people have published a publication here, a real scientific publication 467 00:30:49,820 --> 00:30:52,380 where they did the maths, and they say: 468 00:30:52,380 --> 00:30:55,830 59 people may be dying earlier in the United States 469 00:30:55,830 --> 00:30:59,480 because of the additional emissions in the environment 470 00:30:59,480 --> 00:31:02,530 which they took in and which may damage their body. 471 00:31:02,530 --> 00:31:06,090 The social cost of treating those people — because they may be developing cancer, 472 00:31:06,090 --> 00:31:08,110 they may be going to a hospital, and so on — 473 00:31:08,110 --> 00:31:12,100 is about 450 million Euros. Now that's statistics, right? 474 00:31:12,100 --> 00:31:17,429 "Lies, damn lies, and statistics." Mark Twain is often quoted with that. 475 00:31:17,429 --> 00:31:21,210 But the problem is: That is a real cost, it is a real damage. 476 00:31:21,210 --> 00:31:27,960 If you do violate emission laws it is something that is damaging people's health. 477 00:31:27,960 --> 00:31:31,270 It may be something that is difficult to prove statistically, 478 00:31:31,270 --> 00:31:36,360 but it is something which you don't only do to save money here or there, 479 00:31:36,360 --> 00:31:39,240 it is something which you do to actually hurt people. 480 00:31:40,670 --> 00:31:46,880 Okay, I need to speed up a bit. Very sorry, skip this, that's the next quiz. 481 00:31:46,880 --> 00:31:51,020 15.9 million is actually the salary of this guy here. 482 00:31:53,030 --> 00:31:57,220 That's a lady from BMW, I just wanted to put that out there. 483 00:31:57,220 --> 00:31:59,919 She says, "It shouldn't be called Dieselgate, it's Volkswagen-Gate. 484 00:31:59,919 --> 00:32:02,540 We never did anything wrong at BMW." 485 00:32:02,540 --> 00:32:07,440 And the SZ, actually, yay, they follow, right? In November, it was "Abgasskandal", 486 00:32:07,440 --> 00:32:10,230 in December, it's "Volkswagen- Abgasskandal". 487 00:32:10,230 --> 00:32:15,830 The only problem is that even in 2000, BMW was cought cheating on the Motorrad. 488 00:32:15,830 --> 00:32:21,190 So this is 15 years ago. 15 years ago BMW actually put the same code 489 00:32:21,190 --> 00:32:30,460 which we are now seeing in Volkswagen into their ECUs for the F 650 motorcycle. 490 00:32:30,460 --> 00:32:35,980 And we will see again here the same 34, in this case, -fold increase 491 00:32:35,980 --> 00:32:39,820 in between real use and test bench use. 492 00:32:39,820 --> 00:32:44,559 Now, honestly, they've been caught, they've been caught earlier, and they fixed it. 493 00:32:44,559 --> 00:32:48,690 So in 2001, they actually brought a new version 494 00:32:48,690 --> 00:32:52,720 and apparantly that didn't have this cheat code anymore. 495 00:32:52,720 --> 00:32:56,960 But here we see a pattern again: too little time for development, 496 00:32:56,960 --> 00:32:59,710 too little money willing to be spent on this, 497 00:32:59,710 --> 00:33:01,960 so engineers try to trick. 498 00:33:01,960 --> 00:33:04,260 When you get caught, and you get caught early 499 00:33:04,260 --> 00:33:07,080 nobody probably of you remember this here. 500 00:33:07,080 --> 00:33:09,830 It's fine, it kinda fades away into history. 501 00:33:09,830 --> 00:33:13,929 If you're Volkswagen, if you have 11 million cars out of there, 502 00:33:13,929 --> 00:33:15,600 you have a big problem. 503 00:33:15,600 --> 00:33:20,460 Okay, I'll skip this one, it's really nice, you can see it in the slides. 504 00:33:20,460 --> 00:33:25,809 But I have to go to this here to give Felix enough time. 505 00:33:25,809 --> 00:33:27,940 So how does component development work? 506 00:33:27,940 --> 00:33:34,299 There's a huge set of legal frameworks. It's a very structured top-down process. 507 00:33:34,299 --> 00:33:39,370 You get requirements from the people that represent the market in the company, 508 00:33:39,370 --> 00:33:42,929 you get requirements from the CFO, from the finance director. 509 00:33:42,929 --> 00:33:47,669 And these are broken down into documents which are more than a thousand pages long. 510 00:33:47,669 --> 00:33:53,870 And there's every single detail that could exist in this ECU written out. 511 00:33:53,870 --> 00:33:58,400 There's a piece of paper for everything it does. 512 00:33:58,400 --> 00:34:03,220 Everything. There's not a bit in this thing which is not pushed down 513 00:34:03,220 --> 00:34:06,650 into a very hard set of requirements. 514 00:34:06,650 --> 00:34:12,418 This is then put into a tool, often Rational DOORS by IBM or something, 515 00:34:12,418 --> 00:34:15,109 and then every time something changes this is documented. 516 00:34:15,109 --> 00:34:18,228 There's a complete paper trail, right? 517 00:34:18,228 --> 00:34:20,690 So that means unless there will be a cover-up, 518 00:34:20,690 --> 00:34:23,829 unless we're not given all the information as a public, 519 00:34:23,829 --> 00:34:29,190 there's no way Volkswagen cannot find out who did exactly what at what point in time, 520 00:34:29,190 --> 00:34:31,579 which level of management was involved. 521 00:34:31,579 --> 00:34:36,219 Because every step of the development goes through a Q-Gate, a Quality Gate. 522 00:34:36,219 --> 00:34:39,799 There's managers sitting there and they're approving everything it does, 523 00:34:39,799 --> 00:34:43,059 every progress that has been made, and they're getting reports, 524 00:34:43,059 --> 00:34:45,899 at least bi-weekly, on the progress. 525 00:34:45,899 --> 00:34:51,159 And these reports go up the ladder, they are copied to the next levels of management. 526 00:34:51,159 --> 00:34:55,498 So this is a fully transparent process and this is a fully top-down driven process. 527 00:34:55,498 --> 00:35:00,079 It is completely impossible that you have an engineer that sits there and says, like, 528 00:35:00,079 --> 00:35:05,470 "Well, I wanna cheat," and does the code. There's no motivation for him to do either. 529 00:35:05,470 --> 00:35:11,140 He doesn't get any money for it, he would only be risking his career, so he won't do. 530 00:35:11,140 --> 00:35:14,789 And this is why we have paper trails, and this is why engineers have written down, 531 00:35:14,789 --> 00:35:17,720 "I'm doing this because my manager told me to do this." 532 00:35:17,720 --> 00:35:22,690 And this is why you have Bosch sending a letter in 2007 to Volkswagen which says, 533 00:35:22,690 --> 00:35:27,700 "We delivered you this code you requested. We're your supplier, we do. 534 00:35:27,700 --> 00:35:30,680 But if you send it into production it will be illegal." 535 00:35:30,680 --> 00:35:33,370 And they did. 536 00:35:35,170 --> 00:35:40,989 So this is how actually this exhaust system works. 537 00:35:40,989 --> 00:35:44,109 And this is a little bit important to understand what Felix is now doing 538 00:35:44,109 --> 00:35:50,299 and showing you how the ECU that manages this all works. 539 00:35:50,299 --> 00:35:53,190 To the left would be the engine, to the right is the exhaust, 540 00:35:53,190 --> 00:35:56,920 the end of the exhaust where the remainders come out. 541 00:35:56,920 --> 00:36:02,739 And the first thing is, you have diesel oxid cathalytic 542 00:36:02,739 --> 00:36:06,729 and it basically takes out ... The interesting stuff here is CO, 543 00:36:06,729 --> 00:36:12,569 so carbon oxide, and PM, the particle mass, through 98%, 50%. 544 00:36:12,569 --> 00:36:17,329 The hydrocarbonides before that, they just kind of don't go through 545 00:36:17,329 --> 00:36:21,539 the rest of the process anymore. 546 00:36:21,539 --> 00:36:27,220 Then you have a filter that basically traps all of the diesel particles, 547 00:36:27,220 --> 00:36:29,979 the stuff that causes cancer in your lungs. 548 00:36:29,979 --> 00:36:35,400 But you have to burn them out at some point in time, about every 700 km, 549 00:36:35,400 --> 00:36:36,989 when there have been enough collected. 550 00:36:36,989 --> 00:36:38,460 So it's a bit a trick, right? 551 00:36:38,460 --> 00:36:44,059 The trick is: You collect them so they don't exit the exhaust 552 00:36:44,059 --> 00:36:47,970 but at some point in time you have to burn them again, so they do exit the exhaust. 553 00:36:47,970 --> 00:36:52,729 Now the positive thing here is, they get larger, and the larger they are, 554 00:36:52,729 --> 00:36:58,670 the less risk they — at least as much as we know — cause as a health hazard. 555 00:36:58,670 --> 00:37:04,700 So this is the DPF here. And then at the end, this is the really interesting thing, 556 00:37:04,700 --> 00:37:07,579 this is what most of the scandal now focuses on: 557 00:37:07,579 --> 00:37:10,749 There's a selective catalytic reduction. 558 00:37:10,749 --> 00:37:14,109 And what this thing does is, it does reduce the particle mass, 559 00:37:14,109 --> 00:37:16,099 it does reduce the particles. That's nice. 560 00:37:16,099 --> 00:37:23,239 But the interesting thing is NOx. It goes against this to about 90%. 561 00:37:23,239 --> 00:37:26,650 So this is what it is made for. 562 00:37:26,650 --> 00:37:36,190 It basically injects urea into the airflow and helps to reduce the NOx content 563 00:37:36,190 --> 00:37:41,940 by creating by-products which are mostly water 564 00:37:41,940 --> 00:37:44,809 that comes out the end of the exhaust. 565 00:37:44,809 --> 00:37:47,670 And this is the system, this is a very complex technical system 566 00:37:47,670 --> 00:37:51,839 that has to be managed, and this is managed by an ECU. 567 00:37:51,839 --> 00:37:56,440 This ECU which they selected to do this, and everybody does, is the engine ECU. 568 00:37:56,440 --> 00:37:59,890 Because to the left of the diagram before was this big engine, you didn't see it, 569 00:37:59,890 --> 00:38:04,960 it fell off the diagram, but that's actually the fan blowing into the system. 570 00:38:04,960 --> 00:38:10,729 So this is what you want to manage to actually control what happens there. 571 00:38:10,729 --> 00:38:15,989 Now this thing is quite a sophisticated processor, 572 00:38:15,989 --> 00:38:20,170 it's about the most complex device outside multimedia and entertainment 573 00:38:20,170 --> 00:38:25,400 which we find in the car, and it is a very proprietory thing 574 00:38:25,400 --> 00:38:28,150 because it contains a physical model of engines. 575 00:38:28,150 --> 00:38:32,229 So there have been hundreds, if not thousands of engineers sitting there 576 00:38:32,229 --> 00:38:37,309 and modelling how an engine works, really physically modelling it. 577 00:38:37,309 --> 00:38:41,789 And the things that an OEM — an original equipment manufacturer, 578 00:38:41,789 --> 00:38:44,859 a car maker — can actually tweak are variables. 579 00:38:44,859 --> 00:38:49,049 They can say, "My engine has this and this size, 580 00:38:49,049 --> 00:38:51,890 my combustion cycle looks like this and that." 581 00:38:51,890 --> 00:38:55,309 But the code itself is opaque to the OEM. 582 00:38:55,309 --> 00:39:02,369 It's a proprietory product which you can buy from Continental, or Bosch, or so. 583 00:39:02,369 --> 00:39:06,849 And there's about 20,000 variables which you can tune. 584 00:39:06,849 --> 00:39:12,920 And this thing is simulated and tested to death. Because it is hugely important. 585 00:39:12,920 --> 00:39:17,349 Because you have this machine here that has like 100, 200 horsepowers 586 00:39:17,349 --> 00:39:21,509 and if you steer it wrong it will blow up, and it will blow up really hard. 587 00:39:21,509 --> 00:39:29,039 So this is why this thing is about the best tested piece of software you will ever find. 588 00:39:29,039 --> 00:39:34,729 Which also again means there's everything documented, everything is written down, 589 00:39:34,729 --> 00:39:39,029 everything is seen by everybody who's working with these, 590 00:39:39,029 --> 00:39:41,979 whether it's in development, whether it's in integration, 591 00:39:41,979 --> 00:39:45,720 whether it's in the plants that flash these things, and so on. 592 00:39:45,720 --> 00:39:47,630 There's nothing secret here in this, right? 593 00:39:47,630 --> 00:39:51,319 The functions which are there are actually there to be seen, 594 00:39:51,319 --> 00:39:58,560 well, seen if they are named apparantly, and that is something that Felix will talk about. 595 00:40:00,754 --> 00:40:09,744 *audience applauds* 596 00:40:09,744 --> 00:40:13,190 F: Thank you. Hey, okay. So I will do the second part of this talk. 597 00:40:13,190 --> 00:40:15,029 I'm Felix, by the way. 598 00:40:15,029 --> 00:40:18,400 So my motivation with this was a little bit different. 599 00:40:18,400 --> 00:40:25,749 I'm curious, and, I mean, we can find a lot of source material for this whole scandal. 600 00:40:25,749 --> 00:40:28,249 We can find a lot of information in the press, 601 00:40:28,249 --> 00:40:31,759 a lot of information in the Volkswagen press releases. 602 00:40:31,759 --> 00:40:36,779 However, it should be easier because all the cars are there, 603 00:40:36,779 --> 00:40:42,950 the 11 million cars are out there that have the cheat code in them. 604 00:40:42,950 --> 00:40:47,420 And we are hackers, and we know code, and the truth is in the code. 605 00:40:47,420 --> 00:40:52,660 So my approach was, well, let's take a car, let's take it apart, 606 00:40:52,660 --> 00:40:55,729 let's take the firmware out of it, let's throw it in a disassembler, 607 00:40:55,729 --> 00:40:59,069 maybe get some measurements, and then look at what the car is actually doing 608 00:40:59,069 --> 00:41:04,799 instead of relying on all of this second-hand, third-hand information. 609 00:41:04,799 --> 00:41:06,660 So what do we need for this approach? 610 00:41:06,660 --> 00:41:10,969 So first of all, we need a car that's affected. 611 00:41:10,969 --> 00:41:14,910 You need to drive that car somehow, and driving a car on an open road 612 00:41:14,910 --> 00:41:18,160 can be dangerous if you have to follow particular driving cycles. 613 00:41:18,160 --> 00:41:21,789 So there's a "dyno" you can put the car on and then you can just drive 614 00:41:21,789 --> 00:41:23,599 without the car physically moving. 615 00:41:23,599 --> 00:41:26,309 The wheels are moving, but the car isn't moving. 616 00:41:26,309 --> 00:41:28,069 And this is what other people have done, 617 00:41:28,069 --> 00:41:31,249 and they have taken very interesting measurements out of this. 618 00:41:31,249 --> 00:41:34,170 However, we as hackers, we can go one step further. 619 00:41:34,170 --> 00:41:37,869 We can take a look at the ECU itself. 620 00:41:37,869 --> 00:41:45,469 And not only that, we can also ask other people who worked with these things 621 00:41:45,469 --> 00:41:50,029 and may be able to get more information about them. 622 00:41:50,029 --> 00:41:51,799 I will talk about this in a minute. 623 00:41:51,799 --> 00:41:58,039 So first of all, this is my car, luckily that car was affected by the recall. 624 00:41:58,039 --> 00:42:02,039 So I was very happy when I got the letter telling me I have to go to the shop in January 625 00:42:02,039 --> 00:42:05,559 and get a firmware update because firmware updates are exciting, right? 626 00:42:05,559 --> 00:42:09,969 I love updating things, so updating a car seems great. 627 00:42:09,969 --> 00:42:14,279 Yeah, it sucked that my car was putting out more emissions than it should have, 628 00:42:14,279 --> 00:42:17,599 but otherwise, it gave me the chance to actually look at the car. 629 00:42:17,599 --> 00:42:24,599 I mean, I could have rented a car or something, but that makes it much easier. 630 00:42:24,599 --> 00:42:28,460 I also went on a dyno with my car. On a dyno, there are no speed limits 631 00:42:28,460 --> 00:42:32,509 or no people to run over when you just have to keep a constant speed or something, 632 00:42:32,509 --> 00:42:35,359 so it makes things much easier. 633 00:42:35,359 --> 00:42:39,969 And I talked about ripping apart my car and disassembling it. 634 00:42:39,969 --> 00:42:44,410 I didn't really want to do that, so what I did instead was what I always do: 635 00:42:44,410 --> 00:42:49,789 I go to eBay and I bought an extra ECU. 636 00:42:49,789 --> 00:42:54,929 Here it is, maybe you can show it? 637 00:42:54,929 --> 00:42:58,229 You can go here after the talk and take a look at it. 638 00:42:58,229 --> 00:43:03,989 This is the ECU. This here is the main CPU that also includes the flash. 639 00:43:03,989 --> 00:43:09,509 On the other side there are the power drivers that drive the actual stuff in the car. 640 00:43:09,509 --> 00:43:12,460 And then there's other watchdog circuits and so on. 641 00:43:12,460 --> 00:43:14,760 Okay, thank you. 642 00:43:16,970 --> 00:43:23,180 So, the ECU was built by Bosch, it's an EDC17C46, 643 00:43:23,180 --> 00:43:25,299 that's the name of the hardware. 644 00:43:25,299 --> 00:43:29,099 And it can easily be obtained on eBay, and you can put it on your desk, 645 00:43:29,099 --> 00:43:31,599 you apply 12 volt to it and then it boots. 646 00:43:31,599 --> 00:43:34,279 It will complain about a lot of sensors being missing and so on 647 00:43:34,279 --> 00:43:37,650 but you can see it executing code. 648 00:43:37,650 --> 00:43:43,150 And it doesn't have the very same firmware as my car, but it's very close. 649 00:43:43,150 --> 00:43:46,479 The flash chip is unfortunately in the same pakage as the main CPU, 650 00:43:46,479 --> 00:43:49,519 which is an Infineon TriCore chip, 651 00:43:49,519 --> 00:43:51,779 which is apparantly only used in automotive equipment, 652 00:43:51,779 --> 00:43:55,690 or at least I'm only aware of it being used there. 653 00:43:55,690 --> 00:43:59,380 And I was able to dump the flash by attacking the hardware 654 00:43:59,380 --> 00:44:03,989 and exploiting a bug in the hardware that I haven't found documented anywhere, 655 00:44:03,989 --> 00:44:06,359 but it was not that complicated. 656 00:44:06,359 --> 00:44:09,999 And then I had a firmware dump, I had a 2 megabit binary, 657 00:44:09,999 --> 00:44:12,329 and I throw it in a disassembler. 658 00:44:12,329 --> 00:44:16,910 And what we see is interesting because the code is written very different 659 00:44:16,910 --> 00:44:18,829 from other code that we know. 660 00:44:18,829 --> 00:44:20,930 So usually, code has a lot of flow control 661 00:44:20,930 --> 00:44:24,400 and usually more or less resembles spaghetti code. 662 00:44:24,400 --> 00:44:26,769 This was the exact opposite. 663 00:44:26,769 --> 00:44:31,739 It's more like someone took electrical schematics and put them into code. 664 00:44:31,739 --> 00:44:35,529 There's a set of input signals, there's a set of processing on it, 665 00:44:35,529 --> 00:44:37,009 and there's a set of output signals. 666 00:44:37,009 --> 00:44:42,180 That gets updated every 10 ms or once per rotation depending on processoids. 667 00:44:42,180 --> 00:44:46,759 Really interesting way of writing software and building this. 668 00:44:46,759 --> 00:44:52,279 Also it's very data-driven, so a large part of the firmware is not code but is data. 669 00:44:52,279 --> 00:44:55,829 All of the computations, they don't use constants at all, 670 00:44:55,829 --> 00:44:58,849 they always refer to something from the data section. 671 00:44:58,849 --> 00:45:07,109 As Daniel said, Bosch writes this code, the code is not directly visible to Volkswagen, 672 00:45:07,109 --> 00:45:10,150 but they have visibility into this data, and they know what the data does. 673 00:45:10,150 --> 00:45:12,210 They have tools to change the data. 674 00:45:12,210 --> 00:45:16,839 Volkswagen and other companies can customize this, 675 00:45:16,839 --> 00:45:20,150 really they cannot just customize it, 676 00:45:20,150 --> 00:45:24,089 they can change the whole behaviour of this ECU 677 00:45:24,089 --> 00:45:29,559 by changing just the data, not the code. 678 00:45:29,559 --> 00:45:34,369 The ECU really is a small embedded machine in your car that takes care of the engine, 679 00:45:34,369 --> 00:45:38,849 it's an Engine Electronic Control Unit, there are multiple names for it. 680 00:45:38,849 --> 00:45:42,229 The most important thing that it does is that it takes sensor input, 681 00:45:42,229 --> 00:45:46,180 for example the throttle, and then it applies control to the system. 682 00:45:46,180 --> 00:45:49,420 For example it calculates the amount of fuel to inject, the amount of air to inject 683 00:45:49,420 --> 00:45:54,539 to make the motor running at the speed you want it to run. 684 00:45:54,539 --> 00:45:56,630 These days it's much more complicated. 685 00:45:56,630 --> 00:46:02,400 One important thing the ECU does these days is emission control. 686 00:46:02,400 --> 00:46:06,710 This is why we would expect to find the "cheat code", the code that cheats 687 00:46:06,710 --> 00:46:09,710 that Volkswagen used to cheat in the whole thing, 688 00:46:09,710 --> 00:46:13,430 we would expect to find it in the ECU. 689 00:46:13,430 --> 00:46:16,880 Now taking a look at two megabyte firmware binaries 690 00:46:16,880 --> 00:46:19,769 that doesn't have any visible strings in it, 691 00:46:19,769 --> 00:46:22,880 it's kind of painful if you're just suscepting a code analysis. 692 00:46:22,880 --> 00:46:30,339 So what I did was to do realtime logging. 693 00:46:30,339 --> 00:46:35,279 You can actually read data from your ECU by plugging into this OBD-II port 694 00:46:35,279 --> 00:46:36,999 which is next to your steering wheel. 695 00:46:36,999 --> 00:46:40,180 And while the engine is running you can read out certain data. 696 00:46:40,180 --> 00:46:43,400 Usually you can read out boring data like RPM, and speed, 697 00:46:43,400 --> 00:46:46,640 and some things that the vendor wants you to see. 698 00:46:46,640 --> 00:46:48,910 But there's also a mode that's a little bit hidden, 699 00:46:48,910 --> 00:46:51,089 but you can get pretty easily into it, 700 00:46:51,089 --> 00:46:54,559 where you can read by address, where you can just read the whole memory. 701 00:46:54,559 --> 00:46:58,630 Well, not everything. Some security data is locked out. 702 00:46:58,630 --> 00:47:02,670 But the data we are interested in, we can read that memory. 703 00:47:02,670 --> 00:47:07,569 Now we still need to understand where the interesting stuff is. 704 00:47:07,569 --> 00:47:09,630 We can disassemble the firmware, and that's all fine. 705 00:47:09,630 --> 00:47:13,120 We can also get a little help from something called "A2L files". 706 00:47:13,120 --> 00:47:17,799 The chip tuners use them extensively when they change the mappings, 707 00:47:17,799 --> 00:47:20,640 they want to optimize an engine for a different goal, 708 00:47:20,640 --> 00:47:24,519 for example for more power instead of long lifetime, or something. 709 00:47:24,519 --> 00:47:29,219 They change things in the ECU firmware. 710 00:47:29,219 --> 00:47:34,029 They do reverse engineer a lot, but they also got these files. 711 00:47:34,029 --> 00:47:37,109 And I'm not sure how they got them, but they are out there. 712 00:47:37,109 --> 00:47:40,950 And if you use the right Google terms you will find them. 713 00:47:40,950 --> 00:47:42,920 They are specific to each firmware. 714 00:47:42,920 --> 00:47:45,029 I wasn't able to find one for my actual firmware 715 00:47:45,029 --> 00:47:49,829 but I was able to find one for firmware that is close to mine. 716 00:47:49,829 --> 00:47:53,369 And if you look into this file, what you see is the symbol names, 717 00:47:53,369 --> 00:47:54,749 it's basically a fancy map file. 718 00:47:54,749 --> 00:48:00,940 You see the symbol names, you see a mostly German description of that symbol, 719 00:48:00,940 --> 00:48:06,489 you see a real-use unit, and you see the adress in memory that we can read at. 720 00:48:06,489 --> 00:48:12,489 So with the help of these files we can read out almost any internal state in the ECU. 721 00:48:12,489 --> 00:48:16,650 We still have to make sense out of that, but at least we know where the data is 722 00:48:16,650 --> 00:48:20,420 and what to look for. 723 00:48:20,420 --> 00:48:26,119 It's surprising how complex an ECU is. For example, this thing, what does it display? 724 00:48:26,119 --> 00:48:32,209 Everybody would say it's a function of RPM, it shows you how fast the engine is running. 725 00:48:32,209 --> 00:48:37,249 Well, it's not quite the case, and if we look careful we see that 726 00:48:37,249 --> 00:48:42,180 this code is post-processing the RPM signal. 727 00:48:42,180 --> 00:48:47,019 It's 12 kilobyte of densely written code that has a lot of internal state 728 00:48:47,019 --> 00:48:50,250 that tries to make the RPM value, 729 00:48:50,250 --> 00:48:52,739 convert it to something that the customer wants to see. 730 00:48:52,739 --> 00:48:57,180 For example, you want your idle speed to be stuck at 780, you don't want it to oscillate. 731 00:48:57,180 --> 00:49:01,039 But in reality it does, and this code takes away all of that 732 00:49:01,039 --> 00:49:05,890 and makes it flat 780. 733 00:49:05,890 --> 00:49:10,049 You realize probably at this point that there is a lot of cheating that could go on here 734 00:49:10,049 --> 00:49:12,459 without most people noticing. 735 00:49:12,459 --> 00:49:17,839 You don't really believe that the speedometer in your car displays your actual speed, right? 736 00:49:17,839 --> 00:49:22,190 It displays something related to speed ... 737 00:49:22,970 --> 00:49:25,209 But let's get back to topic. 738 00:49:25,209 --> 00:49:29,349 Selective Catalytic Reduction is the process of, well, if you don't have it 739 00:49:29,349 --> 00:49:34,400 you get a lot of NOx, of nitrogen oxides at the end of the exhaust. 740 00:49:34,400 --> 00:49:36,940 That's bad, you don't want that. 741 00:49:36,940 --> 00:49:42,109 There is one way of getting rid of this, is to add an SCR catalyst. 742 00:49:42,109 --> 00:49:46,099 And the SCR catalyst — I simplified this a lot, 743 00:49:46,099 --> 00:49:48,929 you can find a lot more information about this — 744 00:49:48,929 --> 00:49:56,839 SCR is a process that reduces the NOx using something called DEF, 745 00:49:56,839 --> 00:50:00,599 or AdBlue is a term for it. It's some fluid that you put in there. 746 00:50:04,029 --> 00:50:06,859 Basically it's an Urea/water solution. 747 00:50:06,859 --> 00:50:13,019 And the AdBlue, at a high temperature, converts to Ammonia 748 00:50:13,019 --> 00:50:16,469 and then it reacts with the NOx to nitrogen and water. 749 00:50:16,469 --> 00:50:21,809 Which is great because that's not in any way harmful to us. 750 00:50:21,809 --> 00:50:26,440 However, there's a problem here because the dosage of the AdBlue needs to be correct 751 00:50:26,440 --> 00:50:28,630 and it's very hard to do. 752 00:50:28,630 --> 00:50:33,690 If we dose too little of that the conversion is not perfect 753 00:50:33,690 --> 00:50:36,069 and we will still get a lot of NOx at the output. 754 00:50:36,069 --> 00:50:38,410 Which is better than not doing anything. 755 00:50:38,410 --> 00:50:41,619 It's not perfect, but it's not more harmful than before. 756 00:50:41,619 --> 00:50:45,359 However, if you put in too much of the AdBlue 757 00:50:45,359 --> 00:50:50,089 what you get at the output is ammonia, and you really don't want that. 758 00:50:50,089 --> 00:50:54,769 So the primary goal of emission control is, if you have the SCR system, 759 00:50:54,769 --> 00:50:58,630 is to eliminate as much as possible of the NOx 760 00:50:58,630 --> 00:51:03,170 and minimize the amount of ammonia that comes out of the exhaust pipe. 761 00:51:03,170 --> 00:51:06,640 Ammonia is NH3. 762 00:51:06,640 --> 00:51:10,589 Calculating the right dosage works with a model again. 763 00:51:10,589 --> 00:51:13,459 They modeled everything that happens in the exhaust process, 764 00:51:13,459 --> 00:51:17,390 they have a model of the catalyst, they have a model of the internal state, 765 00:51:17,390 --> 00:51:23,339 they do have a number of sensors and outputs from the other models 766 00:51:23,339 --> 00:51:25,180 that tell them a lot of values. 767 00:51:25,180 --> 00:51:30,349 And the model uses this with a lot of internal storage, internal state. 768 00:51:30,349 --> 00:51:34,619 And the model then calculates the amount of AdBlue to dose 769 00:51:34,619 --> 00:51:42,839 to convert as much NOx as possible without leaking any ammonia. 770 00:51:42,839 --> 00:51:47,180 The way things usually work in an ECU is, there's one system that controls things 771 00:51:47,180 --> 00:51:50,079 and there's another system that monitors things. 772 00:51:50,079 --> 00:51:54,200 It's independent from the main system, it tries to be as independent as possible. 773 00:51:54,200 --> 00:51:57,849 It's still running on the same hardware but it's not sharing a lot of code. 774 00:51:57,849 --> 00:52:04,390 There is an efficiency monitoring scheme that, if the conversion is not good enough anymore, 775 00:52:04,390 --> 00:52:07,930 it will flag this as an OBD-II error 776 00:52:07,930 --> 00:52:10,209 and you will see your "check engine" light going on, 777 00:52:10,209 --> 00:52:13,999 and then you go to the shop, and the shop will diagnose your car and will fix this, 778 00:52:13,999 --> 00:52:18,549 for example if your catalyst is broken. 779 00:52:18,549 --> 00:52:21,749 Based on the test results we would have expected this efficiency monitoring 780 00:52:21,749 --> 00:52:27,219 to actually flag the inefficiencies. But it didn't. 781 00:52:27,219 --> 00:52:30,410 It turns out the main model doesn't always work. 782 00:52:30,410 --> 00:52:33,719 There are some operating conditions where the main model is not sufficient, 783 00:52:33,719 --> 00:52:38,499 it has certain bounds where it works, and outside of these conditions — 784 00:52:38,499 --> 00:52:44,630 for example if the engine is too hot or if the exhaust mass is too large — 785 00:52:44,630 --> 00:52:46,739 the model doesn't produce meaningful results. 786 00:52:46,739 --> 00:52:51,789 It may overdose the AdBlue, and we don't want that. 787 00:52:51,789 --> 00:52:55,450 There's an alternative model which is much, much simpler, 788 00:52:55,450 --> 00:52:58,209 and takes only a few sensory inputs, 789 00:52:58,209 --> 00:53:01,579 and doesn't rely on as many variables to be perfect. 790 00:53:01,579 --> 00:53:04,799 It will still calculate an AdBlue dosage. 791 00:53:04,799 --> 00:53:10,009 However, the main goal of this alternative model is to make the exhaust processing work 792 00:53:10,009 --> 00:53:17,329 in all situations without ever overdosing the NH3. 793 00:53:17,329 --> 00:53:22,549 They're calculating both of these models and then they are selecting one of the models. 794 00:53:22,549 --> 00:53:26,140 The output of the selection then controls the AdBlue dosage, 795 00:53:26,140 --> 00:53:29,269 the pump that injects the AdBlue into the exhaust. 796 00:53:29,269 --> 00:53:34,099 There's code that controls which of the models to use. 797 00:53:34,099 --> 00:53:39,209 There's also a statistics model that counts how often each mode is selected. 798 00:53:39,209 --> 00:53:43,319 Again, all of this model selection depends on the data. 799 00:53:43,319 --> 00:53:45,739 It's code that does the selection but it depends on a lot of data, 800 00:53:45,739 --> 00:53:48,910 there are parameters tought of this. 801 00:53:48,910 --> 00:53:51,769 Let's take a look at the selection criteria for this alternative model. 802 00:53:51,769 --> 00:53:55,200 We see that a lot of these parameters are dummy variables, 803 00:53:55,200 --> 00:53:56,989 things that can never happen. 804 00:53:56,989 --> 00:54:01,979 For example, the athmospheric pressure can't be negative, that can never happen. 805 00:54:01,979 --> 00:54:06,420 Or the air temperature ... I hope it's never larger than that, 806 00:54:06,420 --> 00:54:09,079 or smaller than 0.1K, right? 807 00:54:09,079 --> 00:54:11,859 However, one thing stuck out, 808 00:54:11,859 --> 00:54:17,749 and that was a check if the engine condition is larger than negative temperature. 809 00:54:17,749 --> 00:54:19,839 Which does not exist, the temperature is always positive. 810 00:54:19,839 --> 00:54:23,420 That last one is always true, 811 00:54:23,420 --> 00:54:27,549 so the model that would be selected would always be the alternative model. 812 00:54:27,549 --> 00:54:30,150 That sounded weird and I was looking at the firmware. 813 00:54:30,150 --> 00:54:34,640 Maybe I understood it incorrectly, or maybe I looked at the wrong place 814 00:54:34,640 --> 00:54:37,630 when looking at these parameters? 815 00:54:37,630 --> 00:54:42,369 But if we look at the intermediate results there is a bit at a certain location 816 00:54:42,369 --> 00:54:47,579 that tells us which model was selected, and that bit is indeed always set. 817 00:54:47,579 --> 00:54:51,249 That is weird, it sounds fishy. 818 00:54:51,249 --> 00:54:57,489 Let's take a look at the statistics, the car counts what model you're in. 819 00:54:57,489 --> 00:55:00,309 20% of the cases my car does not do dosing at all. 820 00:55:00,309 --> 00:55:02,630 So I drove some time and then looked at the values. 821 00:55:02,630 --> 00:55:06,369 And the 20% where it doesn't do anything is mostly the warm-up cycle. 822 00:55:06,369 --> 00:55:09,469 But everytime it does something, it's actually the alternative model 823 00:55:09,469 --> 00:55:14,719 which we know does underdose NH3 because it doesn't want to leak ammonia. 824 00:55:14,719 --> 00:55:19,449 And that makes sense because my car uses much less than expected of the AdBlue. 825 00:55:19,449 --> 00:55:25,529 The expected value is roughly 2.5 liters per 1000 kilometers, of the AdBlue. 826 00:55:25,529 --> 00:55:28,779 In my case it only used 0.6 liters per 1000 kilometers. 827 00:55:28,779 --> 00:55:32,469 Which is great for me because I don't have to refill this tank very often. 828 00:55:32,469 --> 00:55:37,319 In fact, I never had to do it, the shop always does it when I'm there. 829 00:55:37,319 --> 00:55:42,380 But this is fishy, and let's take a look at this. 830 00:55:42,380 --> 00:55:46,029 What we also see is that sometimes the regular model is active, 831 00:55:46,029 --> 00:55:47,859 so there must be something more. 832 00:55:47,859 --> 00:55:52,539 If we look at the selection criteria we find that there's an additional term there 833 00:55:52,539 --> 00:55:54,959 that I haven't found before. 834 00:55:54,959 --> 00:55:57,049 There's an additional condition that has to be true 835 00:55:57,049 --> 00:56:01,680 in order to go to the alternative model that underdoses. 836 00:56:01,680 --> 00:56:04,989 We look at the particular conditions and we find a lot of stuff 837 00:56:04,989 --> 00:56:07,739 that is related to diagnostics, things they can do in the shop. 838 00:56:07,739 --> 00:56:09,969 So that's definitely not happening on the street. 839 00:56:09,969 --> 00:56:12,959 But one of the criteria, that really was weird 840 00:56:12,959 --> 00:56:18,769 because it looks if the engine and fuel temperature is larger than 50°C, 841 00:56:18,769 --> 00:56:24,249 it looks at the athmospheric pressure and if it's lower than 750m, 842 00:56:24,249 --> 00:56:25,809 that must be satisfied. 843 00:56:25,809 --> 00:56:29,859 If all of these conditions are satisfied it will move back to the main model 844 00:56:29,859 --> 00:56:33,759 that does the proper exhaust processing. And one thing was really weird. 845 00:56:33,759 --> 00:56:36,380 There were seven curves, not all of them used, 846 00:56:36,380 --> 00:56:38,789 that define an upper and a lower bound 847 00:56:38,789 --> 00:56:42,799 on the distance driven after a certain amount of time. 848 00:56:42,799 --> 00:56:46,459 This is how it looks in disassembly. I'm not sure if you can read this. 849 00:56:46,459 --> 00:56:52,699 But the comments are from this A2L file and they call it "acoustic function". 850 00:56:52,699 --> 00:56:55,440 I'm not sure if this has anything to do with acoustics. 851 00:56:55,440 --> 00:57:00,900 I tried to find all the usages, and there was nothing related to sound or anything. 852 00:57:00,900 --> 00:57:04,180 I think it's just a name for it. 853 00:57:04,180 --> 00:57:09,579 Now if we go and take a look at these upper and lower bounds, we see this: 854 00:57:09,579 --> 00:57:16,999 These are three curves that are defined, each of them has an upper and a lower bound. 855 00:57:16,999 --> 00:57:19,259 It's basically the distance 856 00:57:19,259 --> 00:57:22,670 that you need to have driven after a certain amount of time. 857 00:57:22,670 --> 00:57:27,479 And if you ever fall out of one of these curves we're switching back to the alternative model 858 00:57:27,479 --> 00:57:30,859 that underdoses NH3 and causes the inefficiencies. 859 00:57:30,859 --> 00:57:34,109 This is weird, and I didn't really know what this is. 860 00:57:34,109 --> 00:57:38,709 Let's get back to something completely different, which is the NEDC. 861 00:57:38,709 --> 00:57:43,219 We've seen this slide before, the NEDC mandates you how to drive. 862 00:57:43,219 --> 00:57:46,479 One thing is also interesting: It mandates you that ... 863 00:57:46,479 --> 00:57:50,269 You want this test at "cold-start", and what's better for a cold start 864 00:57:50,269 --> 00:57:54,660 than heating the car to 20°C and keep it that warm until you start. 865 00:57:54,660 --> 00:58:02,819 That's the "cold-start", that's the cold start as defined in the law: 20°C. 866 00:58:02,819 --> 00:58:10,339 This is speed over time, so to get distance over time we need to integrate this. 867 00:58:10,339 --> 00:58:12,869 And we get this graph. 868 00:58:12,869 --> 00:58:17,479 And if we overlay what we found in the firmware we get this. 869 00:58:17,479 --> 00:58:27,140 *audience laughs and applauds* 870 00:58:27,140 --> 00:58:31,660 What we can see here is that if you drive the driving cycle correctly 871 00:58:31,660 --> 00:58:35,660 you will exactly be in the bounds of one of these curves. 872 00:58:35,660 --> 00:58:37,400 And you can do this on the street, you can do this everywhere. 873 00:58:37,400 --> 00:58:42,779 As long as you satisfy the distance over time and your car is warm enough 874 00:58:42,779 --> 00:58:46,609 it will detect this in some way. 875 00:58:46,609 --> 00:58:49,650 Well, you can drive this on a street, but it's really dangerous 876 00:58:49,650 --> 00:58:52,690 because you have to follow a given speed pattern. 877 00:58:52,690 --> 00:58:55,799 So i did this on a dyno, I put my laptop in there, 878 00:58:55,799 --> 00:58:58,849 I logged the data in real-time and then displayed it. 879 00:58:58,849 --> 00:59:02,959 Basically, this is what it looks like. In the middle you see a bar. 880 00:59:02,959 --> 00:59:06,469 You have to drive and keep this middle bar in the middle, 881 00:59:06,469 --> 00:59:11,410 which means you are well within this upper and lower bound, and not try to escape it. 882 00:59:11,410 --> 00:59:17,099 And as long as you do, one of the other green boxes will tell you 883 00:59:17,099 --> 00:59:22,249 that the car is still detecting this as being in this cycle. 884 00:59:22,249 --> 00:59:29,539 Then what I did in the end, I stayed in the cycle for a while and I logged all the data. 885 00:59:29,539 --> 00:59:32,089 At the end I would just hit a constant speed 886 00:59:32,089 --> 00:59:36,130 which would eventually get me out of the conditions. 887 00:59:36,130 --> 00:59:39,509 This is the log that I made. 888 00:59:39,509 --> 00:59:41,880 On the first graph you see the vehicle speed, 889 00:59:41,880 --> 00:59:45,049 you see how I tried to follow the NEDC more or less successfully. 890 00:59:45,049 --> 00:59:49,559 On the second graph you see the distance over time, 891 00:59:49,559 --> 00:59:54,680 you see that I stay within the bounds enforced by the firmware. 892 00:59:54,680 --> 00:59:56,739 You an also see on the third graph— 893 00:59:56,739 --> 00:59:59,179 this is the actual signal at the AdBlue pump— 894 00:59:59,179 --> 01:00:02,130 that it actually doses quite a lot of AdBlue. 895 01:00:02,130 --> 01:00:05,630 It calculates the amount of AdBlue to dose based on the model output 896 01:00:05,630 --> 01:00:09,420 which you see in graph 5 and 6. 897 01:00:09,420 --> 01:00:14,839 By the way, graph 4 is the actual NOx emitted by the engine based on their model. 898 01:00:14,839 --> 01:00:20,789 That's the RML, their mission model then calculates the amout of the dosing to happen. 899 01:00:20,789 --> 01:00:25,880 As we see, as long as we stay within the limits enforced that match the NEDC 900 01:00:25,880 --> 01:00:29,579 everthing is good and a lot of AdBlue is dosed. 901 01:00:29,579 --> 01:00:32,749 And then, in the end, I drove too fast. 902 01:00:32,749 --> 01:00:35,979 And you can see in the second graph that I crossed the upper bar, 903 01:00:35,979 --> 01:00:38,249 the blue line goes over the red line, right? 904 01:00:38,249 --> 01:00:40,989 You can see that the car immediately detects this, 905 01:00:40,989 --> 01:00:44,529 that I'm no longer in the driving cycle. 906 01:00:44,529 --> 01:00:52,259 The interesting part you see here is the effect on the AdBlue dosing, which is here. 907 01:00:52,259 --> 01:00:57,619 It immediately stops doing the dosing. And you can see in the model below 908 01:00:57,619 --> 01:01:01,670 the model still calculates that AdBlue should be dosed. 909 01:01:01,670 --> 01:01:04,630 But after they have the max, after they switch the model 910 01:01:04,630 --> 01:01:08,709 and switch to the alternative model, the alternative model just outputs zeroes, 911 01:01:08,709 --> 01:01:11,869 it doesn't dose anything. 912 01:01:11,869 --> 01:01:14,359 This shows that when we're following the cycle 913 01:01:14,359 --> 01:01:17,539 everything is fine, enough Urea is dosed, 914 01:01:17,539 --> 01:01:24,599 and then once we leave the cycle, there's a severe reduction in the dosing. 915 01:01:24,599 --> 01:01:26,949 And it's all based on detecting this driving cycle. 916 01:01:26,949 --> 01:01:29,180 Two more slides. A: Two more slides. 917 01:01:29,180 --> 01:01:32,349 F: Two more slides. A: Two more slides, here we go! 918 01:01:32,349 --> 01:01:38,299 *audience laughs and applauds* 919 01:01:38,299 --> 01:01:40,979 I have to be clear on the limitations here. 920 01:01:40,979 --> 01:01:43,599 All of this was looking at disassembled code and so on, 921 01:01:43,599 --> 01:01:47,559 I could have done something wrong here, so take this with a grain of salt. 922 01:01:47,559 --> 01:01:50,979 We couldn't do NOx measurements on the dyno, unfortunately. 923 01:01:50,979 --> 01:01:55,569 And I have to stress: We looked at one particular car that uses SCR processing, 924 01:01:55,569 --> 01:01:57,660 not all of the affected cars are doing this, 925 01:01:57,660 --> 01:01:59,719 there are some other mechanisms in the other cars. 926 01:01:59,719 --> 01:02:02,559 And I looked at a car for the German market, 927 01:02:02,559 --> 01:02:05,569 at least the curves have to be different for the other markets. 928 01:02:05,569 --> 01:02:10,900 Let's reenumerate the results— and this is my last slide. 929 01:02:10,900 --> 01:02:16,749 Most of the time, on a regular car, a nonstandard treatment mode is active 930 01:02:16,749 --> 01:02:20,799 that is not as efficient as the real mode that is implemented. 931 01:02:20,799 --> 01:02:23,309 We can show the code that is responsible for this: 932 01:02:23,309 --> 01:02:26,269 This is this negative temperature limit that they look at 933 01:02:26,269 --> 01:02:29,809 which doesn't make any sense and always selects the alternative mode. 934 01:02:29,809 --> 01:02:32,630 And we can see, in the logs, the state selection bit, 935 01:02:32,630 --> 01:02:38,359 we can see the counters that count that the alternative model is active. 936 01:02:38,359 --> 01:02:42,019 We can see that there's an AdBlue underdosing in this state 937 01:02:42,019 --> 01:02:44,920 which causes the inefficient NOx conversions, 938 01:02:44,920 --> 01:02:49,609 that's what we've seen before when people put the car on the dyno. 939 01:02:49,609 --> 01:02:52,939 We know that the efficiency checks are only enabled in the main mode 940 01:02:52,939 --> 01:02:57,019 and the car does exceed the limits. 941 01:02:57,019 --> 01:03:03,400 This shows how the alternate model is selected where it doses too little AdBlue 942 01:03:03,400 --> 01:03:06,569 and causes the inefficient conversion. 943 01:03:06,569 --> 01:03:10,209 We can see that if we follow the driving cylce, 944 01:03:10,209 --> 01:03:12,459 the minimum temperature and the distance over time, 945 01:03:12,459 --> 01:03:14,869 we will see that it switches to the main model 946 01:03:14,869 --> 01:03:17,640 that should have been active all the time. 947 01:03:17,640 --> 01:03:20,289 We can show the code that's responsible for that, 948 01:03:20,289 --> 01:03:23,529 the driving cycle detection that uses the upper bound and the lower bound. 949 01:03:23,529 --> 01:03:28,630 We can extract the exact limits, overlay the NEDC data and see that there's a match. 950 01:03:28,630 --> 01:03:33,680 We can, if we do this actually on a dyno, we can see how it switches the SCR state. 951 01:03:33,680 --> 01:03:37,319 We can show the effect on the DEF dosing, on the AdBlue dosing. 952 01:03:37,319 --> 01:03:41,499 As you've seen on the slide before, as soon as we switch out of the driving cycle 953 01:03:41,499 --> 01:03:48,170 into the street mode, the dosing will get close to zero. 954 01:03:48,170 --> 01:03:50,910 Once you're back in the main model all the efficiency checks are enabled, 955 01:03:50,910 --> 01:03:54,519 for example to take better Urea. 956 01:03:54,519 --> 01:03:56,979 So the efficiency checks are there, but they are not active 957 01:03:56,979 --> 01:04:00,680 because the car is forced to run in the alternative model. 958 01:04:00,680 --> 01:04:04,709 These results are all in line with the Volkswagen press releases. 959 01:04:04,709 --> 01:04:07,859 These are basically just the details as extracted from the firmware 960 01:04:07,859 --> 01:04:10,450 to show you the background. 961 01:04:10,450 --> 01:04:12,330 Thank you. 962 01:04:12,330 --> 01:04:16,190 *audience applauds* 963 01:04:16,190 --> 01:04:20,229 A: Wow! Thank you very much, Daniel and Felix. 964 01:04:20,229 --> 01:04:35,919 *audience applauds* 965 01:04:37,609 --> 01:04:40,269 I'm really sorry, but we have to clear the stage. 966 01:04:40,269 --> 01:04:42,609 There is not going to be time for the Q&A session. 967 01:04:42,609 --> 01:04:46,650 Do that down there. I'm sure that a few people just come down, 968 01:04:46,650 --> 01:04:49,670 grab you and ask questions. Unfortunately, we can't do that. 969 01:04:49,670 --> 01:04:54,059 I have to close it in exactly four seconds over here because we have to go off the stream. 970 01:04:54,059 --> 01:04:57,529 Thank you very much Felix, thank you very much Daniel. 971 01:04:57,529 --> 01:04:59,190 F: Thank you. 972 01:04:59,190 --> 01:05:10,161 ♪ *postroll music* ♪