*wikipaka preroll music*
Hello and welcome to my talk on Remixing Linux. My name is The one with the braid,
and today I'm going to introduce your own creation of a Linux distribution. We will
talk on remixing Linux, on the architecture you require to run an own
Linux distribution, and some of the usual use cases. OK, let's start. Well, what are
use cases, what are purposes of your own Linux distribution? Of course, you could
customize Linux distribution for your enterprise environment. For example, if
you require special network settings, your own services to be installed, it's
sometimes easier to create an own distribution than using a ready to use
distribution and customizing it. Another option is that you discovered something
fully new and decide to create your own Linux distribution. That's what we call
yet another Linux distribution. Please, please do not do this. We already
have enough different flavors of Linux. OK, another valid reason is missing
software. If you require a special software for use case and you want to
bundle it into your operating system. That's a usual use case. Moreover,
embedding for microcontrollers is a common use case for Linux remixing, as well as
OEM hardware support for computer vendors to bundle Linux. Some known vendors do
this. Yeah, and of course, another desktop or in general different preinstalled
applications are a good reason to create your own distribution. OK, what are we
going to have a look on today? We will look on live systems, because that's the
most common use case. Afterwards we will go through step by step instructions for
and next, we take an example of Ubuntu. We talk on making your system installable and
we will talk about a simple tool simplifying some of these steps. We will
have a look on Linux from scratch, meaning compiling everything yourself without
using any templates or anything. And we'll look at the architecture you require to
run your own Linux distribution, a.k.a. package mirrors. OK, let's start with a
live system. What are properties of a live system? Of course it needs to fulfill some
requirements. If you use a tiny pen drive, I don't know, eight gigabyte or 16
gigabyte, it's obvious you need to compress your whole system. Moreover, it
must be writable at runtime. Just try to mount your root filesystem in Read-Only and
run a web browser. You run into big trouble. Same for network connections. If
you connect to wifi without writable root system, you will run into big trouble. And
another requirement, it's not really a requirement, but it's recommended is an
accessible live media. So if we have our pen drive containing our live Linux
distribution, we sometimes need to access the pen drive itself in our live
environment. For example, if we'll have packages of our distribution inside or on
our pen drive, we may want to install them later. Then it's quite good to have your
live media accessible. Well, of course, we could script everything on our own, but
someone already did all these steps and has put them into a tiny software called
Casper. Casper does exactly what we want, creates a union file system, meaning an
overlay on the existing file system and creates a kind of fake writable system in
memory, file system in memory. So it allows modification of files, creation of
new files while they are actually being stored in the memory instead of on our
physical disk. It mounts the live media and it shows up a pretty display manager
you can see over here. That's the Ubuntu mate, it asks whether you want to try our
system or whether we want to install our system. OK, let's talk about the
compression, because the compression we require is tremendous. If we have a normal
desktop distribution. Believe me, it consists of about 30 gigabyte of data.
Our applications, as soon as you install some more applications like LibreOffice or
a Web browser and some tools, you will reach the 30 gigabyte or 20 gigabytes, but
if you ever downloaded a live distribution, you will realize it's not 30
gigabyte. It's about two gigabyte or one point five. I noted in my slides. Yeah, it
means we need to compress our file system on about 5 or 10 percent of the actual
size. That's not difficult. There are many compression algorithms we may use, but we
have another requirement. We need performant readability. As soon as we boot
a system, our kernel won't stop accessing any files, reading files, writing files.
So we need a performant readability because no one wants to wait five minutes
to see the Web browser. And that's what the squash file system is made for. The
squash file system is a compressed file system. It's kind of a disk image, but more
advanced. It's used in diverse Linuxes. It's used in the operating system of
Chromecast and in the Linux userland it's used by app images. These are these click-
to-run Linux binaries you can download, and it's used by snap packages from
canonical's proprietary snap package manager, which is unfortunately very
widely spread. Yeah. Now we know something about compression. Let's talk about how
the customization works. Well, first of all, we download a template. We create a
base system from which we start our customization process. Well, we start
the customization itself. We prepare a CD environment. So when you put a bootloader
on a CD, we pack our root file system we just created, put it on the CD, put a
kernel there, make it, harden it a bit, and at the end we will pack it. OK, let's
go through it, step by step. In this case, we will have a look at Ubuntu, we will go
step by step through the customization process of Ubuntu. How do you start the
customization process? Well, you need an existing ISO file from a live environment
of Ubuntu, because it contains the bootloader, the directory structure and
everything, all the assets. Of course, you could create them on your own, but, well,
it's more work than you have to do. You can easily use the existing bootloader and
the existing files and directory structure from an existing ISO file. Well, let's
extract it and we're fine. We're done with it. Afterwards, we need to create a base
system. A base system is, well, the base of Linux. We have the base utils, the
base files, the directory structure, the kernel, a shell. The package manager of
our operating system we have. Without any unnecessary tools, without, just the base
of everything. You can either download the prepared base system of the distribution for
download, but you could create one on your own using tools like the other Debian tool
to create base system or the corresponding tools from other distributions. Yeah, as
soon as you have your base system, you need to change your root inside this base
system. And then you start the customization process by installing the
necessary drivers. We already talked about Casper. Casper is this tool providing the
live software and the live scripts, the overlay file system in Ubuntu based
systems. And it creates an initial RAM- file-system and installes the required
kernel modules to load and to boot a squashed file system. So let's install
these three packages, Casper, Ubiquity- Casper and Lupin-Casper. And afterwards we
can install whatever we want. We can install our loved vim, we can install our
loved Alpaka software, and we can install an alternative shell, the packages we
want. In the example code I provided here we look on changing the plymouth screen.
That's the startup screen which shows up when you boot the distribution. I don't
know, you could put an Alpaka in there or the logo of rC3. Yeah, that's what we do
in customization or modification of the system. Afterwards, we continue packing
everything we created. First of all, we need a Kernel to boot , because well
before our base system can be extracted, we need to load a kernel by the bootloader
and the kernel must contain the corresponding kernel modules to load, to
boot a squash file system. That's what Caspar already prepared for us, we only
need to create the initial run file system and if you read the code I provided, you
will see I was missing an minus between update and init RAM file system. Let's
create our initial RAM file system and copy them or copy it together with our
kernel to our extracted live environment. And can delete the prior kernels and so
on. We don't need them. We have our own kernels now. Afterwards we pack our base
system. Of course we need to unmount any mount points in the base system we created
and create a squash file system from it. And these three files, the VM Linux, the
initial Ramdisk and the squashfs file system, they are the major pillars of our
live environment. If we have those three, we are actually ready to go now. We can
either use our existing bootloader from our extracted ISO file or we, of course,
you can install your own bootloader in this directory. But well, it's easier to
use the existing. For security purposes and for hardening your live environment I
would recommend to put the MD5 sums of all files in your ISO file in this
environment, because as soon as you, for example, copy your distribution, for
example, on a pen drive on which is fat or ext4 formatted, it could be modified by
anyone. And that's why it's much better to verify the MD5 sums at the boot process.
Casper will automatically detect whether the MD5 sum, the txt file exists and if
it exists, it will automatically verify these MD5 sums. OK, now we are actually
ready to go, ready to pack up our ISO file. There's a famous tool called
genisoimage. So generate an ISO image, that's a command you could use. The
command takes care of legacy booting except of output file, the disk and the
disk image and the -r parameter, you could leave out everything if you do not require
legacy bootloaders. But if you want to obey legacy bootloaders, you should also
hybritize your ISO file to be able to dd the ISO file to a pen drive. Otherwise,
without the ISO hybrid command, it won't boot from, it would boot from a CD, but
not from a pen drive. Yeah. OK, now we created our own Linux distribution. Well
not that much. It's actually just a repacked Linux distribution. Now we talk
about making it a real distribution. That's why we talk about making it
installable. Why should we make it installable? Of course we could manually
install our system. We could manually format our hard drive, extract our file system and
manually create users and so on, et cetera, et cetera. But most of users
prefer graphical way except of these geeks taking part at the rC3. That's why an
installer may be useful. An installer does exactly what we need. It creates
partitions, creates users, installs language-packs, offers installation of
extra software and does everything the user might want. Creates users, connects
to wifi, whatever. And there are three very common installer frameworks:
Ubiquity, Calamaris and Anaconda. Now we'll have a look at these frameworks.
Ubiquity is known from Ubuntu. It's ready to use if installed, but it's actually not
at all customizable. Anyway, some hackers at GitHub found some ways to customize it
and provided some good information, some helpful information in their repositories.
The next one is Calamaris. Calamiris is highly customizable, is used by many
distributions, including SUSE Linux, I think Xubuntu, so the XFCE Ubuntu flavor,
DeepinOS, I think I am unsure. I think elementary as well. But if you only
install it, it won't work. You need to customize it. The third one is known from
the Fedora project. It's called Anaconda. It's a very advanced tool. It's able to
update existing systems. If we have an old Fedora installation, we can boot our live
CD and update the existing installation and moreover, very useful for admins is
the remote functionality. It can be started via SSH and automatically update
everything and automatically install and deploy the software we need. Um, yeah.
Well, I already mentioned, for everything you can manually do and customize, there
is a graphical tool. If you prefer the graphical way you could use the tool
remastersys, aka LinuxRespin. That's the the older name. It offers a simple
graphical user interface for packing your currently installed system into a live
distribution, allowing to install it. It does not allow a high amount of
customization in the graphical way, of course, you can manually perform it on the
command prompt. But it's an easy way to, for example, backup your just installed
system or you readily configured system to a pen drive to make it installable later.
And now, Linux from scratch. I guess most of you already heard of Linux from scratch
from scratch means, yeah, pretty much Linux from scratch. You start without
anything. You start on your host system and create an own system from scratch.
What does that mean? It means you should have lots of time. Lots of RAM, and many
processor cores. Because you need to compile everything yourself. Beside the
kernel, you need to compile the whole userland. If you want, you can include a
package manager or whatever. But if you make real Linux from scratch, you do not
take any binaries but compile everything yourself, everything on your own.
Currently, minimally, you need to compile 226 dependencies. But in some years it
will be some more dependencies and the amount of dependencies required increases.
Yeah, after you compile everything, of course you need to configure everything,
need to provide a bootloader. But as soon as this is done, as soon as you created
your own base system, your own system from scratch inside a fake root, it's the same
as we already talked on during live system customization. As soon as you have this
fake root, you simply pack it to an image, so a squash file system, write it on a
disk, provide a bootloader, and that's it. OK, one more word. I will talk about
maintaining architecture for distributions. I will talk about package
servers. If you have a distribution, you usually provide updates, security fixes
and whatsoever. Maybe software the users may install later. Yeah, that's a package
mirror, a packet server. From Debian, you can simply mirror existing servers, for
example, the Debian or the Ubuntu or the Linux Mint servers by using the tool apt-
mirror. It simply mirrors, means it does not allow any customization. For ArchLinux
it's easy as well. You can simply use vftp and crontab to download the packages and
provide them on your host system, on your server and synchronize from them. That's
much easier than in the Debian environment. But usually if you have a
distribution, you want to customize it. At Debian, the tool aptly is very widely
spread because it allows very advanced management of different snapshots from
different repositories, mirrors, local repositories, single packages, allows
diffs and so on. But it does not allow any optimization. So good luck at scripting on
your own. If you look at ArchLinux, for example, you would usually write your own
package build files as you know them from the Arch user repository and include the
readily build packages into your directory from which you serve your updates. That's
much easier than in the Debian environment. I still found one more
graphic explaining aptly. That's what aptly can do. It can do everything. Well,
you have mirrors, you have local repositories. You can take snapshots, take
differences from snapshots, publish these differences, publish merged snapshots of
them, publish single snapshots, diff everything and do whatever you want. It's
very performant, the tool and yeah, but it's hard to use. It requires some time to
be learned. Yeah, that's it already on remixing Linux. Now there should be an
Q&A. Thank you very much for your attention of this talk. I hope I could
give you a good introduction into remixing Linux and good luck at coding and enjoy
your Remote Chaos Experience.
*Music*
Subtitles created by c3subtitles.de in the year 2021. Join, and help us!