This subtitle is not finished yet. If you are able to, please support us and watch the talk in amara for the last changes: https://c3subtitles.de/talk/1360 Thanks!
Herald: Hello and welcome to Infrastructure Review. This review is
being translated into a lot of languages and we don't know yet which one, but the
c3lingo team will be on stage and will tell us how and what it did. I'd like to
start, as always and every year, with the NOC, right. So please give the NOC a hand.
*applause* [Filler, please remove me in amara]
Momo: All right, everyone, welcome to the State of the Internet manufacture report.
This is JC, I'm Momo and we're going to talk to you about what we did this year
for the network. So obviously, organizing Congress is a quite tedious task. Took us
about six months of pre-planning. We came in on the 15th, did a fiber day and then
it took us from the 18th and will take us till tomorrow to tear everything down with
like 20 to 40 people. And we'll be busy wiping every device because that is
actually what we do every year. We delete everything. There are no logs leaving this
building and this will take us probably the next 24 hours. So yeah, for the usual
numbers. Edge capacity: this year because you didn't use all the Internet last year,
we only brought you 300 gigs, but that was fine as well, I suppose. We got 100 gig
from HLKomm here in Leipzig, 100 gig from Deutsche Telekom and as well as 100 gig
from BCIX which we got via DWDM wave to Berlin. In the core we used Juniper
MX960s, MX480s, MX204s and QFX10002 in the yolocolo. Basically all the halls were
connected via a 200 gig link and yolocolo had three times 100 gig. As probably the
last five or so years, we're using IS-IS and BGP for our protocols of choice. And
this year we also first off rejected RPKI invalid routes and secondly applied for
the first time at congress BCP38 ingress filtering to be a good internet citizen
and not to allow you to spoof IP addresses. So yeah, that was nice. As
Congress keeps on growing, we have 2500 tables somewhere around the building in
all the assemblies, but we only have 300 switches. So sorry if you had to bring a
long cable and if you have switches to spare with 10 gig uplinks and POE+, feel
free to donate them to us. Access and Wi- Fi. We had like 300 access switches. We
are obviously again running Aruba Wi-Fi controllers. This year, like at camp we
had a few 802.1x access points, more on that later. We tried to use Juniper vMX to
route the Wi-Fi traffic. And had quite a shitload of switches, most of them from
Juniper, some Cisco 2960s, some Brocades which are new to us, and some crappy old
HPE stuff which is basically configured for us to work like a brick you get from
like eBay or whatnot. We had a few incidents this year we'd like to talk
about. First off, we had, I'm not sure if any of you noticed, quite a lot of packet
loss and missing router advertisements on the Wi-Fi. This was caused by some weird
Juniper vMX behavior. We couldn't figure out what it was. So we had them running in
a redundant VRRP setup. We shut down one of them and then it worked. So yeah, fuck
redundancy. There was a pixelflut client which somehow messed up his IP address and
caused a broadcast storm which took down most parts of Hall 2. We found them, shut
it down and deployed storm control to all our access switches. Yeah, to the Congress
motto resource exhaustion: someone was running aggressive zmap scanning over the
whole internet, came by our Wi-Fi access controller and caused a state table
exhaustion. And that brought it down. We null-routed the source and yeah, there was
this issue. So thank you to whoever was that. And in the morning of day 3, we had
another issue with Juniper vMX where it forgot it had a network card. We rebooted
it and everything was fine again. So yeah, some numbers. You actually managed to use
more bandwidth, thank you. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
M: But it's still only 20% of our uplink capacity. So use more. 20% of that was
IPv6, which is good, but could be more. We had like 11000 clients into Wi-Fi. 86% 5
GHz, 96% in a peak. We had eleven 802.11ax clients. Our
favorite one was obviously the one with the lovely hostname ILOVETHENOC. So yeah,
about that number we have, 96% 5 GHz obviously shows us that we are finally at
the point where we can say: thank you 2.4 GHz, it was nice. Goodbye.
*applause* [Filler, please remove me in amara]
M: Also, obviously, thank you to our sponsors. We couldn't do this if we would
not get like 10 millions, of list price obviously, of equipment and loan and quite
a lot of services. So give them a round of applause as well. Thank you.
*applause* [Filler, please remove me in amara]
M: And obviously NOC not only stands for Network Operation Center, but if you
extend it, it is No CO2. So we believe in green power and clean traffic and
therefore we obviously see that sustainability is a great part of our
role. This is why we even use old crappy HP switches to cut our lines for our
tchunk to serve your cheese boards, whatever you need.
*laughter* [Filler, please remove me in amara]
M: Also... [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
M: Also, we somehow estimated what our network will run us in CO2 and that was
about 11 tons. We're not very good with mass and not very good with CO2 emissions,
but this was roughly what we came up with. And to make Congress or the world a bit
better place, we actually offset all our CO2 and bought eleven tons of CO2
emissions. And now Frederick is gonna tell you how we got all those numbers.
Frederick: Yeah, as a good internet manufacture, we also do monitoring a lot
and we run our own Prometheus server inside. You probably know the dashboard
that we propagate all over the internet and that's powered by Prometheus. We have
an internal Grafana that is part of this whole ecosystem. And if you are a little
bit of a nerd, you might have clicked on the dashboard sections and seen that there
are more dashboards than this. We fill our Prometheus with lots of different sources:
we get SNMP data from, and screw SNMP, but it does a quite good job at getting all
the insights we need from all the network equipment. We have node_exporter, influx
and all that, but we got a decent amount of data from everyone in the Congress
ecosystem and we had that at camp as well where we got the water pressure of the
showers. And we get the colo power, which also helps with estimating the CO2
footprint. And everything is being configured by Netbox, which is a tool that
is an asset database. And as I said, we have lots of dashboards and graphs. Of
course, the public one where you can see lots of different things from everyone.
This is only part of it. If you scroll down on the dashboard, you see a lot more.
But this helps everyone to have a good understanding of what is happening
currently. And we even draw a nice little Christmas trees on the Wi-Fi traffic for
you. That is mostly because it's a router- on-a-stick and we cannot measure it
correctly. We have an internal dashboard which gives us a little bit of a status
for build-up mostly: which switches and routers are up? And that gives us a very
quick sight of all the devices that are out there. What's broken? What's not
broken? We improved it a little and now have alarms so someone can look at stuff
and see if things are broken, run out there and fix it. We also built
weathermaps. As you can see that's a little bit of a mess. But we couldn't do
it better because the graphing library doesn't allow us to do it better. If
someone has a good idea to do it better in Grafana or anywhere else with sources from
Prometheus, please come to us, we're happy to talk. But this shows our core and all
the links between it and how much capacity is being used. Red indicates that it's
used more heavily. We also have that for the yolocolo. And all the traffic around
it as well. And, yeah, teardown starts now. Please don't touch our equipment. And
if you want to come and help, please come to Hall 4 and get in touch. We always need
helping hands. But please in an organized way don't disassemble switches or access
points. We have lists and everything. We need to account for everything. So please
come to Hall 4 if you want to help. And yes, use more bandwidth and offset more
CO2. Thank you. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: So thanks a lot. Actually, because you're clearly the backbone or one of
those many backbones of the conference, is there any Q? Let's do a Q&A for like one
or two questions. Is anyone having a question right now?
Someone is standing up. Right, microphone number one, please.
Q: Hey, we've absolutely don't ... [Filler, please remove me in amara]
H: Nearer, nearer. [Filler, please remove me in amara]
Q: We've absolutely done bcp38 in previous years, by the way.
M: Sorry for what I said then, I'm sorry. Q: So I wanted to correct the record.
We've been good netizens in previous years as well.
H: Oh, you're right. Thank you. Was there another question or something else to
correct them? Because they clearly don't know what they're doing.
*laughter* [Filler, please remove me in amara]
H: Yeah. Feel free. Microphone number two. Q: So if 2.4GHz is over, what's going to
happen to all of the ESP32 in various IoT devices? Are they going to have a home
here next year? M: They'll definitely have a home. But as
usual, we cannot support it as good as we can on 5GHz because obviously this band is
overused and not even remotely suited for that amount of clients we put on it.
Q: A follow-up for the ESP32. How exactly can you locate them through the wireless
if they are lost? M: Well, we can't. We can basically look
at which access point they are, and then if someone really would want to, we could
start triangulation, but we've never done that before. So yeah, we can just pin them
roughly to an access point. H: Maybe we can ask c3nav next time. So
please give the NOC a hand. Thank you.
*applause* [Filler, please remove me in amara]
H: So, the next team up is the POC. Do we have to click this? Use more bandwidth,
I'm going to try. Ah! Thank you. Your stage.
Garvin: Thanks. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Maria: Hi, my name is Maria. [Filler, please remove me in amara]
G: Hi, and I'm Garvin and we're from Phone Operation Center and we want to talk a bit
about the phone infrastructure at this event.
M: Yeah. So we arrived at day -6 and planned on hacking some things
and socializing and we planned a team event, but then everything was different.
G: Yeah. When we arrived, I went into the NOC office and they said to me, "Ja, CCL
is up, internet is up, everything is just working nicely. You can start hook up your
telephony system right now." And I was like, whoa. So thanks a lot NOC. Really
great performance this year, we were amazed. Nobody expected that it works so
well. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
M: Yeah. So we put up the first antennas and then we decided to have our team event
anyway. And yeah, so we did a lot of things. So we handed out 150 orga loan
DECTs and we deployed 51 SIP telephones. We also deployed 67 antennas and we had a
POC party on day three until 7:30 a.m. G: So you can see almost everything is
done. I guess the remaining things are not that important. So this is the overview of
DECT coverage at the event. Only level zero, because otherwise I think it would
be overcrowded to show. Just so that you can get a rough impression on how many
antennas we deployed in order to give you this DECT coverage. That you can be
reached almost everywhere in the event and that you can see how our tooling looks
like, where we see how good the antennas see each other, and that we can see that
seamless handover work so that you can start at our desk, walk through the area,
into the lounge and just continue talking. And oh, there are also some antennas that
are outside of the building. What could that be?
M: That is our hotel DECT. And you can see a typical hotel DECT installation on the
photo. And people got really confused about it because we also had DECT coverage
at main station. G: Yeah. So I got a call roughly at 4:00
p.m. in the morning. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
G: And somebody told me. Oh, my DECT rang. Why? Now I need to turn it off at the
night. Why did you do this? M: Yeah. So we also had problems. We have
a new feature since Camp where you can see your DECT devices and can assign them to
your number before you even arrive, and then everything is set up and you don't
have to call your token anymore. And people get really confused because they
would call their token anyway and it calls it invalid. So we had to explain a lot of
this unexpected simplicity to them. G: And then we had a battery issue this
year. We had not enough batteries and, you know, batteries are always empty in the
phones when it's the most important. So we were thinking what what can we do about
that? M: So we build a new device. That's our
microwave and it can also charge devices. So, many thanks to C3Power, because they
helped us with tooling and actually they have expertise to put power cables on
devices like this. Thank you very much. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
G: In the last years we were often asked "how expensive is your service actually?".
So we decided that we now provide invoices so you can see how expensive our services
are and we send out a lot of invoices. And we got paid some money. But as you can see
on the invoices, most is sponsored by CCC. M: Yeah. And people also paid with Mate
which is really awesome. G: Yeah. And also people have invoices on
a fraction of a cent and they got quite creative on how they can pay us.
*laughter* [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
M: So here are some more stats. We have 7473 registered extensions.
G: Oh, we didn't remove this. So we were thinking on how to compare this with
things and we were looking at villages in Saarland and then we thought, this is a
stupid comparison, but we didn't remove this.
M: So there were 5021 attached DECT phones and 3251 concurrent DECT phones. Which is
about more than 1000 more than last year. G: So thanks a lot for using DECT.
M: Yeah. We had... [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
M: There were also max 120 calls in parallel, and we had more than 300
thousand calls in total. That's also really, really much. We had five eating
meetings at heaven, the angel eating place, and there were an average of 42
eating meeting live viewers. We had two lectures. You can see them on media.ccc.de
and we had 23 super fast charged phones in our microwave.
*applause* [Filler, please remove me in amara]
G: Like the NOC, we also had to deal a bit with issues during the event, and actually
there were some DDOS attack on our account system and somebody configured over 4000
extensions with really stupid names. And it took us quite a while to get rid of
them again cleanly from the system because the synchronization turned out to be
really slow. So you can see it took us a while to get them removed again. So we can
only say, you all know it's a hacker congress, but it's kind of stupid to hack
your own infrastructure. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
G: So as a consequence of this, we only allow now only 50 extensions per account
and per event. If you think you need more, feel free to contact us if you have a
valid use case. [Filler, please remove me in amara]
M: So every device has an IPEI that is like a MAC address and we ask the
responsible institution to give us the manufacturer. But it's really secret. So
they don't give it to us. So we ask you for help. Please enter the models of your
devices for your phones. And then we can match to the IPEI and get some data to
build more awesome features for you. [Filler, please remove me in amara]
G: This would really help us. And the only way for us is to crowdsource it because it
seems to be super secret. Whatever. [Filler, please remove me in amara]
M: You can find it in guru3 on the device page and there's this little pen. And if
you click on it, then you can enter the model.
G: Thanks upfront. And yep, that's from us, and I guess now we have a little time
for you to ask questions. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: Great. So, any questions for the POC? I don't see... were they correct or do we
have to...? Ah, someone is getting up. So microphone number 1, please.
Q: Yes, a few years ago there was a translation service via DECT. Is the
capacity enough to service also this crowd?
G: The problem is that we switched the phone system a while ago last year at the
Congress. And the old phone system had a way how we can do the translation via one
channel and the problem is that the new system doesn't support this. Let me say
the new system doesn't support it yet. So have a look at our talk and then you can
see that there is some potential. H: I see someone at microphone number 3,
please. This would be the last question because we have to hurry a bit.
Q: Can I know a little bit more about the super charging microwave? I'm confused.
*applause* [Filler, please remove me in amara]
M: Sure, you can come to our POC desk and then we answer all your questions.
H: Ooh... mystery. Please, give the POC a hand.
So next team, is it from the GSM crew? Someone there? I think we have two. I see
some Chaos Post, you have to wait one round. So, GSM guy thank you. I am good
with computers, I think. Yeah, I am. Have fun. If it's working or not. Look at this
smile then looks better. Try this. Yeah. Right. Try this.
- 423. - Maybe you have to use GSM.
- 23 test. - What happened?
- Hello. Hello. - Aah.
GSM Person 1: So as every year we ran our own mobile phone network at the Congress
using osmocom open source software for 2G and 3G, and open5gs interfacing with the
osmocom HLR. And all you need to take part is a SIM card that you can buy from the
POC and for 5 euros you get a flat rate. The price increases because we have less
SIM cards every year. We need to manufacture new ones. You can even call
outside like you can with DECT phones. Lynxis: Hello. I'm Lynxis. So as every
year for the GSM team, the first problem is the license. That's the first step
usually. Because in Germany, you have to get the official form, get a license but
... Where do you get it? What can you ask for frequencies? Because, for example, the
POC for DECT or Wi-Fi, you just place it and you can use it. You're fine.
But for GSM, they didn't think about it or for 3G or 4G. So yeah, this year we
usually get the license middle of december, maybe start of december. So it's
already late. So this year we didn't get all our licenses. But we get some. We got
850 MHz, which is not assigned in Europe because it's usually in the US only. But
we have a small hole. This year we got a 4G license instead of a 2G license with 10
Mhz from Telefonica. So thanks Telefonica for borrowing us spectrum.
*applause* [Filler, please remove me in amara]
L: Just a short example of how the spectrum looks like. The yellow stuff is
usually somewhere behind actually. By the way, this microphone, it might share the
same frequency with us. But so far, we haven't found any interference together
with the VOC. Down there you can see the small hole which we are using. Because we
didn't get the 2G license there, we thought, OK. Let's take a look. Can we fit
them both in the same frequencies? It's not good. But you see the spikes, this
nice antenna on the right. That's the GSM, which is sending on the same frequency. It
works because they are using different codings. But I have heard from people who
know more about it, this is not the way you use it.
*laughing* [Filler, please remove me in amara]
L: So we took some photos of our base stations. This is actually the fairy dust
from the 2G base station, so we have a idea what we are using here. We have even
more fairy dust in our 3G femtocells and our 4G cells. They are looking like small
toasters. They are taking actually 90 watts via POE, they have special POE
adapters. So maybe we could ask if somebody can do a similar adapter to get
even running a toaster on the line. So basically for the 4G setup this year,
we weren't sure if it's stable enough or we lose all our phones to the LTE and they
don't like to come back to the 2G and 3G setup where we have voice, because on LTE
we don't have yet voice so you have to select specifically to join the LTE
network. That worked quite fine if you change it. So your phone will register.
Everything fine there. GSM1: So the rollout this year: we had the
voice working on day 0 which is new. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
GSM1: Someone even noticed on Mastodon, I saw it, too. We already had LTE at the CCC
camp this year. Yeah, but unfortunately we lost crypto password, so LTE roll out took
a bit longer this time. Sorry. So some numbers. In total we saw just about 1100
people doing a location updating on our network and 845 eventphone tokens were
dialed on the GSM. That is 2G or 3G. And there were roughly 200 phones actively
subscribed on the network at all times. And even though we basically only deployed
3G nano stations in all the halls and only had two 2G BTS in the glass halle, there
were roughly more than half of all the phones were still subscribed on 2G instead
of 3G. We had like 18 3G stations and only two plus one in the GSM room 2G stations.
So that's a bit surprising. And SIM cards: starting from the bottom, we sold about
700 SIM cards, but only saw half of them activated, which is curious. And luckily,
most or some people bring old SIM cards from previous years. And it's not so easy
to get cards manufactured. So we are very glad for everyone who brings old SIM cards
from previous events. We might even consider introducing charging phone calls
only for SIM cards that are newly bought. So if you want to continue calling for
free, rather bring your old SIM cards. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
GSM1: So you see the numbers, I'm not going to go through them. And you can also
get the ADM, the admin keys for your SIM cards if you like to write to them.
L: Or if you have seen the talk from LaForge, you might want to play with the
SIM cards, we give out all the keys you want to have to play with it.
GSM1: So operation was mostly smooth, except iPhones, for unknown reasons. And
except the data service, which might even be related because maybe Apple is a bit
more strict on whether data service is working reliably. Yeah, we still had some
problems in the SGs instance introducing 3G changing between the radio access
technologies. It's a whole new ballgame so there are still some bugs in there. And as
you see, we had many more tickets than the POC. This is actually reversed from the
POC, the "done" is on the left. So this whole bunch of stuff is done and there's
some backlog and canceled and fantasy tasks. It worked pretty nicely. Are you
taking over? Oh, yeah, no, this is still mine. And this year we actually had also a
denial of service attack. The code was the same as previous years and we never saw
this before. But this year we got an invalid mobile identity which managed to
crash our mobile switching center. And thank you very much for uncovering this
bug and thanks to fixeria for fixing it on day 2. Ever since the mobile switching
center for Voice and SMS and subscription has been running stable.
*applause* [Filler, please remove me in amara]
L: Maybe some interference. We recovered the old phones again. Last year we
couldn't support them. But we managed to implement the missing parts. Old phones
could work if they support the frequencies. So that's really nice. Maybe
next time. Since camp, we also did a nice angel helpdesk. And it was really
impressive to see that we even had to add more shifts in our shifts. We had so many
motivated angels. Thanks to everybody who helped us, it was really great.
*applause* [Filler, please remove me in amara]
H: Unfortunately, we don't have enough time for a Q&A. So please give them a hand
for everything they did. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: Thank you. There are quite a lot of teams next. I count at least like eight,
maybe nine. So we need to speed up a bit. Our next team for now will be... We don't
have working microphones. Chaos post 1: Sorry, we need to interrupt
you anyway. H: OK. So tell them, chaos post! Chaos
post! [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Chaos post 1: Sorry, guys, we interrupt... [Filler, please remove me in amara]
H: Let's get rid of the broken one because it's not working anyway.
Chaos post 1: All right. Sorry, we are interrupting for a few minutes only. We
would like to deliver a few statistics as well. So thanks of all, we had multiple
chaos deliverers working throughout the whole Congress 24/7 basically, delivering
at the speed of chaos, as our mission statement clearly states. So thank you
therefore, first of all thank you very much to all of you who contributed to
that. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Chaos post 2: So let's have some numbers, because you all love numbers, right? So we
delivered about 3000 external postcards. So that means with like outside chaos, so
to the real world or the default world, as we call it. We delivered those to over 42
countries all over the world. So you guys are really good connected internationally.
And also, we don't have exact numbers, but we estimate around 3500, no, 35000,
internal postcards. And you also use our online office for a total of 789 times. So
that is only 15 less than camp, and that was a longer event. So, hey, you guys
write a lot of postcards! [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Chaos post 1: As you might already know from camp, we also do have a few special
services. As already on camp, we had the serving proposal, basically a pre-
assembled text, you just had to cross what you want. The postbox certification and of
course, the bi-directional chaos, in Germany also known as "Einschreiben mit
Rückschein". H: Wow.
Chaos post 2: And this time we also offered some new services. We had like
sang telegrams, gesunges Telegram in Deutsch. We had a forever alone box for
people wanting to write postcards with or exchange postcards with no idea who to
write it to. We had love letters, so we had some nice pre-assembled texts and also
a really nice selection of perfumes for scented postcards. You could write some
secret messages. We had some UV pens and also we had some, let's say call it
security, or rather temper evidence, because we had some scratch off stickers
for you. And also we had over 30 new postcard designs that you could use for
postcards. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Chaos post 1: Short remark for the love letters and the perfumes. Well, that was
kind of not really thought through. It was a bit... it was fun sorting them out, and
stamping all of that, smelling all the perfume all the time.
Chaos post 2: And then brushing your hands really thoroughly because, well, that
stuff gets on your hands when you do that. Chaos post 1: Also for UV pens. Just a
little remark. It's not a good idea to use it for addressing and the message.
*laughing* [Filler, please remove me in amara]
Chaos post 1: You can take the risk. All right, then let me close up. We also
supported mail this year and we had 130 letters for activists in
prison, which I find really great. I think that's something we can all support.
Amazing. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
So thank you all for this amazing event, and have some fun for the rest of the
Congress. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: Chaos post! So the next team is the VOC. Yes, you can have it. Here, the VOC.
We only have 20 minutes left in total for every team. So you know what to do.
VOC1: Hurry up. Welcome, guys. So. Yeah. We don't have as many statistics as usual.
But we have some great stories too. We'll hurry up. So this year in total, we
covered 10 stages apart from the 5 stages that we do usually for Congress. We had
streams from the critical decentralisation cluster, Sendezentrum, Wikipaka WG, Open
Infrastructure Orbit, Chaos West. And in total, we served 255 hours and 35 minutes
of total talk time. So you know what to do until the next Congress.
*applause* [Filler, please remove me in amara]
VOC1: And of course, sustainability was a big topic during this Congress. So part of
what we have to do is stream reencoding so you can watch it with the VP5 codec or use
it at lower resolution. And so far we've been using 4 Xeon-based machines and 2
desktop machines. And thanks to hardware- based encoding, we now replace this with a
single laptop. As you can maybe read, this is critical infrastructure now. And for
all the streams, for 30 reencoding streams, we are on a 45 watt power budget
now. And as an added benefit, because we also encode the master slides with
hardware encoders, and hardware encoders can generally use a higher profile that
allows for better quality in real time, you now get a better picture as well. So,
yeah. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
VOC1: And of course, we had some minor fuckups this year. We thought the audio
setup was a bit less complex. But when we checked rooms, we had buzzing everywhere.
So we replaced some SDI lines with fiber and turns out buzzing goes away. Then the
PA sound console had a buggy DANTE card. After you reboot them, the auxiliary out
to the VOC cameras was muted. So that messed up the particular talk that has
been redone today, but other than that, we are rather happy. And we figured that 3 of
our audio mixers actually had broken outputs. I don't know how we did that, but
it clearly shows that they've been used on one event or another during the last
years, which is actually a good thing we may think. On the Wikipaka stage, we did
not use Ansible because there was sort of a playground for us. But if you don't do
things properly, well, then you run into edge cases with things. And yeah. So I
need to hurry up. And one virtualization host suddenly started leaking memory. And
so if you were affected by that during the main talk season in the evening, we are
very sorry about that. Updating the kernel helped and we have no idea what happened.
Yeah, and Icecast got stuck as well. And some relive... so if you want to see talks
later, that may not be possible because we ran temporarily out of space. But if you
watch this on media.ccc.de, and the talks were not yet released, we have relive
integration, so the talks show up in media.ccc.de even though there is no
proper release yet, for your convenience. And main track and assemblies are now
integrated in all events, so you don't have to click through 4 separate events to
find your favorite talk. And now I pass to Pat to talk about VOCTOMIX 2.
Pat: Yeah. Thank you. Okay. I have now 20 seconds, I think. I made a redesign of
VOCTOMIX, it's now called VOC2MIX. And Peter was doing that meme some weeks ago,
because we had to switch to VOCTOMIX 2 and we wanted to try it in 2 rooms. And in the
night from day 0 to 1 we decided to do it for every stage because the old solution
was not working anymore. That was a little bit hot, but it worked. And, the redesign
caused the new UI, complete with some new base features. We have now transitions
where pictures are moving, and we have insertions for blending text into the
picture and we have a new audio mixing and we are now proper A/V Sync in every case,
I think. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Pat: And some mixer angels were exploring the software and they found some bonus
features like random video distortion in some cases, which I have to fix, I think.
And the party mode where you click some buttons and they are clicking without any
doing of the user and everything is flickering and they called it party mode.
This is what the current pipeline looks like. We have now over 200 gstreamer
elements doing all this stuff to get your pictures, which you are having on the
stream and in the recordings. That's it. VOC1: So, of course, there were some
issues. I mean VOCTOMIX2 is essentially a rewrite.
H: You have to speed up a bit. Actually we are stealing time from the other teams. I
might call it a quit. VOC1: OK OK OK. So...
H: Pressure! VOC1: Yes, I know. But you're not making
things better. So one thing to mention: we had to deploy a sweaty finger fix after
the first talk started. Ok, here are your stats, read them now, read them read them
read them! [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: Please give VOC a hand. Thank you. So what's next? Oh, the Stage manager and
Herald Operation Center. I'm part of it. And we have 36 heralds, every one of them
very eloquent and good-looking. Then we have 70 stage managers and stage
supporters. We had 150 shows on official stages and the assemblies on top of it. We
have one stage fright council yet for the speakers who took care of at least six
talks. And then we threw away over 100 hosting cards on day 1 only. So clearly,
give a hand for the SHOC. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: So next, c3power. Your time. Arif: So my name is Arif Guy. I am from
the power team and from the radio operation center. We make the power. We
load five double trucks of shit out on Sunday 15th. We deploy a lot of power
boxes and many cables. So we have teardown today so every help is needed. We have
only one day to bring us back. The main thing, we had the power meter that we made
on the camp this year. We have two nice setup on room H. You see the power factor
is very bad. And the other one was the Waffle Operation Center so you see they
have a nice power factor. So please use more ohmic devices like heaters, waffle
iron or something. We only have 5 Seaview to monitor Yolocolo. You have on
c3power.top a Grafana link that also links to the main Grafana to the NOC. Another
nice thing we have on the lounge, I have a video here. It started now. You can see
the current on all 3 phases to the audio and you can see the audio from the lounge.
*recorded music from the lounge* [Filler, please remove me in amara]
Arif: It's very nice. [Filler, please remove me in amara]
*recorded music from the lounge* [Filler, please remove me in amara]
Arif: As you see, the only thing is... [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Arif: Use more bass, or make more current. Later we can show you another slide. Go to
next slide please. So for the radio team we have 120 portable radios. We have
updated them to a new firmware, and a new programming software with a new feature
that is a lot slower than the last version. Very nice. 50 bring-your-own-
device radios. 5 dead radios from the camp. 2 dead from 36c3. 2 dead repeaters
from the camp. 1 dead on arrival on this stage. 3 rental repeaters from 2
companies, one we picked up in Hannover just before the Congress, and even the
windows PC crashed down dead on arrival. Business as usual. So, next team!
H: Thank you! So, the next is c3subtitles. I think it's Amy and Julia. Amy. They are
not. C3SOS. S.O.S. It's you. Okay. Okay. Sorry. Oh, I'm sorry. Not the subtitles.
It's well, sustainability. Your stage, go for it. Here, feel free.
Amy: Hello. Okay, hi. My name is Amy, I'm part of the c3sustainability team. I only
have four slides, so I'll be very quick. You can see, the first of our biggest
projects was the drinking water dispensers. So for angels, we had drinking
water from a dispenser rather than from bottles and they could refill their
bottles. We go through some stats there. So when we started planning how difficult
it was to implement it, the locations, and how many volunteers, satisfaction was very
high for this one. So thank you to c3geld who really helped organize the water
dispensers with us. We're very happy. They were really great. Please give them a
round of applause. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Amy: Thank you. The next one was a give and take electronic box. So we wanted to
encourage people not to throw away their electronics. Of course, they can be
recycled. So we have deployed 10 boxes. You can find them in the sticker stations.
And we did this in collaboration with the hardware hacking area. Thank you. And
other assemblies. Oh, wow, you are really fast.
H: Yes I am. Amy: Okay!
H: Others are waiting! Amy: Okay. So the last one I wanted to go
through. We have two initiatives on this one. I'm sorry for the trashy picture
there. We have organic bins in the halls. It was very, very difficult to do this.
But actually it was quite satisfactory. But I would say there is room for
improvement there. And there was also an initiative for recycling cigaret butts. So
we actually had two people go round, collect your cigaret butts and they will
be recycled into cool ashtrays. What a success! Thank you so much to everyone who
collaborated with us. We couldn't do out without your help.
H: Thank you. Your applause, please. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
H: So unfortunately, the c3sign is not coming. So just take a look at those
pictures while I go through them. And now the next one is C3 Assemblies. Here, take
this one. Thank you. Be quick. Pingu: Hi, my name is Pingu. I'm here for
the assembly team. And I just want to ask, give a hand who hasn't found his assembly
on day zero or day one? So then give a big applause to c3nav because they really
helped us a lot. Because without them, it wasn't able to do this event just for us.
Because for some figures we had 419 assemblies to place on this area which is
about 35000 square meters. We had 3000 tables all over, 2500 in the assemblies,
with 6000 chairs. And here, please give a big applause to the C3Möbelhaus or IKEA as
you call it. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
Pingu: Because the Möbelhaus basically placed all the tables in a magic night on
basically day -2. And they will disappear with the help of C3Möbelhaus today. We
assigned the last assembly on day zero at 22:00 and we started our work in mid-
October with weekly meetings. And yeah, as you can see, it was a lot of work and...
Oh. H: Thank you.
Pingu: Thank you. And just one thing, for teardown, for tearing down the assemblies,
please stack the chairs on the assembly but leave every chair and every table on
the assembly, we will get rid of them. H: Thanks a lot. Your applause. So the
next ones are... Les prochains, ce sont les gens de c3lingo, voilà, vos tours!
c3lingo 1: Hallo! Schön euch mal (...) Wir sind übersetzet. (...)
German> Oh, sorry. Well, I'm also fine with that.
We translated all the German talks into English. All the English talks into
German. All in all, fifteen thousand minutes in seven different target
languages. And here is from the second language team, who talk all the non-
English and non-German things. c3lingo 2: Okay. So basically we did have
about 1/3 of the talks were translated to French. Yeah. You can read the rest. And
we even have currently, exactly, right now, another one which is translated into
Swabian. So if you want to listen into it. [Filler, please remove me in amara]
*applause* [Filler, please remove me in amara]
c3lingo 2: Which means that if you were listening to a talk, there is 2 chance out
of 3 that it was translated not only once, but twice. Into either French, Spanish,
Russian, or Polish. One special mention for the Russian and Polish teams, that was
their first time this year. So one big round of applause for them, please.
*applause* [Filler, please remove me in amara]
c3lingo 2: And one more big thank about for the guys who brought the cough candies
and cough drops in the booth. That's a lifesaver. Thanks.
H: Thank you. Merci beaucoup. So we have some heart operation going on here. You
have to switch it or not? Oh, you're good with computers.
*laughing* [Filler, please remove me in amara]
H: He just *mimicks something*. So c3infrastructure from the subtitles. Your
stage. Yeah. My microphone. Ha ha. td: So, thanks. Just a quick look into the
subtitles. So what do we actually do when we subtitle a talk? Well, first of all, we
take the video from the C3VOC and put it through speech recognition just to get a
rough transcript that we can then give to angels to actually correct, because, well,
machine speech recognition doesn't work so good at all. And then once we have a
working transcript that humans have looked at, we put that through auto-timing, which
just takes the transcript and aligns that with the audio, and that usually works
pretty well. And, well, once that is finished, then we actually have working
subtitles, but we give them to angels for another round of review just to fix any
mistakes that got overlooked. And maybe sometimes the timing needs to be adjusted.
And then, when that is done, well, the subtitle is released. And actually one of
our angels did a nice chart about all this process that you can see here. It all
sounds better, so thank you for that. Well, no presentation without graphs. As
you can see, the important thing is really: everything goes up and to the
right. On the bottom here we have finished seconds of transcribed talks. So this is
really completed subtitles here. And it starts already quite high because it
includes all the Congresses before. Then we have stuff that has been reviewed in
orange. Stuff that has been timed but not reviewed yet in yellow. And transcribed
but not yet timed stuff in green. All in all, we had 144 distinct angels. Yeah, I
need to hurry up. 71% of which took 2 shifts and 10% took 7 or more shifts. So
433 hours of work for 126 hours of material. And so far we've had 6 releases
from this Congress and then lots of hours worked. All of these numbers are at least
as high as last year's numbers, so good, thanks. When you have transcripts, you can
do cool stuff with that. So, for example, generate word clouds to see what people
like. And this case, people seem to like people and questions and time, which we
don't have any here. So. Well, how do we actually keep track of all this
complicated stuff? Well, we use a state- of-the-art NoSQL lock-free columnar data
store, like many of the other teams also do. And well, thank you all of our angels
for your hard work. And also thanks for the Heaven for supporting us. And then,
well, if you if you feel bored between Congresses, you can still continue to work
on transcripts. You have all the informations here, these slides will be
online, follow us on Twitter. And thank you.
*applause* [Filler, please remove me in amara]
H: Please do your thing again, like your thing again, like do *mimicks
something*. Take your laptop. [Filler, please remove me in amara]
*laughing* [Filler, please remove me in amara]
H: So this was the last one. I think I will try my best to do something like this
too. Actually I'm not good with computers, but I know someone who is and who takes
very great care of everyone of us. So one of my highlights of every Congress, feel
free, the LOC! LOC: Hello. I'm the stand-in for LOC. As
with all good projects, they're too busy for documentation. They're packing. So LOC
doesn't have anything. I'm more of the Department of Health and Safety again. So
for everyone: we have the message from CERT that there were no work-related
accidents that caused real harm. The odd broken Mate bottle maybe. But thank you
for having built a city again safely and orderly, even with all the chaos. For the
people who are driving, please make sure that the drugs wear off and that you get
some sleep and for all people riding along, please keep the guys awake. That
would be greatly appreciated. Go home safely. Thank you very much.
*applause* [Filler, please remove me in amara]
H: So. Wow, ah, OK. So I'm very sorry to have to rush some teams and I'm very sorry
that we don't have any time left. We are 1 minute over and I promised the teardown
crew to not do overtime. So please, please give all the teams their respect and clap
and tramp as loud as you can for now to finish!
*standing ovation* [Filler, please remove me in amara]
H: Everything! They did! For us! We did! For them! Why are you still sitting? You
have to leave. Thank you. From the c3infrastructure Review. Goodbye.
Subtitles created by many many volunteers and the c3subtitles.de team. Join us, and help us!