Dear viewer, these subtitles were generated by a machine via the service YouTube and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/2830 Thanks!
hi welcome back everyone um as usual few
bits of information first so if you want
to interact with us you can do so by
sending us questions requests or memes
on madon um by using the hashtag Shan 20
sh23 um you can talk to us in the IRC
chat on ic. hack.org in channel shunks
23 um we've got a uh social viewing in
the aquarium and jitsy the link can be
found on our homepage fire.org
um we have a YouTube and a twitch
Channel which we stream too uh each at
fire shuns um those two uh channels also
have a live chat which we will also
monitor for questions um our main stream
streaming channel is at streaming.
media.ccc.de under the name fire shs
with real
life today I've got uh Amy spark with me
they are a they are a software
development contractor Jack of many
trades when it comes to desktop app um
in a past life they specialized in
material appearance modeling high
performance Computing and imaging today
they're talking to us about the unsung
heroes of Imaging um this talk will
introduce lumac chroma color spaces what
they are the reasons for existence why
they are useful and real world
examples Hey all good afternoon my name
is Amy spark and the title of this talk
is the unsung heroes of
Imaging for a bit of background about
myself I work for the British company
Centric as a map stream and compiler
breaker we are F pveers of free and open
source software Consulting and our
primary responsibility is the
maintainership of the G streamer
multimedia
framework throughout the years I
specialize a lot in cor spaces s
language build systems in short anything
that can be csed and I have also
contributed on occasion to a lot of
projects most related to color modeling
Sim and so on all of them free and opers
Source software
so as a motivation for the reason for
this talk I bet that most of you have
heard of terms like jpg picture PNG some
other folks may have also used the T
file format if you have watched TV you
may have also heard about stuff like pal
ntsc ccam so
on so the question is how do we store
and broadcast color
images so for inance inance why is a
picture stored in the JPG file format so
small compared to say the PNG equivalent
the T equivalent the row equivalent that
we get as a row format from the
cameras another question is how can we
watch those cut videos on pH on phones
and not get blown away by the
bandwidth another fter question is how
were we able to watch color TV before
the Advent of the PCS in our homes
the answer to all of these questions is
that there is a lumac chroma color space
involved to compress them there is lots
of background to cover so I hope you are
all ready and for those that live on the
C of the way be sure that this also HDR
covered in
stock this stock is divided roughly into
four sections in the first T section I
will explain the very basics of a color
spaces
in the second section I will explain a
lot of about lumac chroma color spaces
what they are and what they can be used
for in the third section I will explain
a lot of examples that are currently in
use and some that that were part of the
anent story of computer science and in
last section I will summarize the thing
that we have seen and make some remarks
about the stuff that can be covered in
the
future so with our f
let us start with the introduction to
the world of lumac chroma color
spaces who are colors
specified to process the light that we
see around us into a color image that
you can later store or transmit or
whatever we need to start thinking of
colors as coordinates in a color
space just put it a bit more formally a
color space that and dimensional
geometrical model for instance three
dimension
that turns light stimulate that is to
say colors into Vector coordinates back
and forth for instance if you have
already taken a picture with your camera
or phone or if you have browsed the web
you have already run across color
spaces that is because the srgb color
space Powers most of the consumer
content
nowadays to put it in a more formal or
mathematical footing a color space is a
coordinate
system on which you define a
Subspace along with a mapping function
from a supported color to a single point
inside a top
space and finally the set of supported
colors that you obtain is what we call
the color space gamut or
range to give a formal or standardized
definition of a color space you need two
set of
components
first you need to specify the primaries
that is to say three independent
reference
stimul and two you need to specify the
color of the light source that we call
the white point or the
illuminant this components are represent
by what we call chromaticity
coordinates and plotting them into what
we call chromaticity diagrams reveals
the color space gamut for instance on
the right we have the chromaticity gamut
one of the cor space that we will see
later called bt79
ICPC the vertices of the triangle are
the three independent primaries the
little dot in the middle is the
coordinates of the light source or
luminant in this case
d65 and the whole area of the triangle
represent the gamut that the IC can
cover now there is a bit of a misnomer
of of some inconsistency between books
as regards the storage and transmitter
of color data and to solve that we need
to go from the abstract to a more
concrete definition if we do so we have
color model
types then color
models then color spaces we are here and
finally the most complete implementation
which is color and
coding I'm going to make a stronger
emphasis the first item color model
types and the rest will arise easily
through some examples that I will show
you in the next few
slides the type of a color model tells
the practitioner how the primaries
represent colors we have additive color
models that work by mixing light Sim to
form colors the most well know example
is RGB where you have three primaries
one representing the red color another
representing the green color and finally
another one representing the blue color
color and you have three coordinates in
the vector RGB for each
Channel if you have worked in web
development you may have seen linear
transformations of RGB such as hsl HSI
or
HSP and the other type of color model is
subtractive which works by absorbing
light in order to generate
colors the most wellknown one of course
is a model that all in printers work
with which is called CMYK foran marenta
yellow and black
ink it should be noted that substractive
colors models are not covered in this
talk so I'm not going to talk further
about
them so for an example I told you
earlier that srgb or standard RGB poers
most consumer content
nowadays it's what we call an additive
color space of the AR V
model it has a standardized definition
as an ISO a standard publish
2002 and it has a very specific encoding
in general most the standard will tell
you the conversion formulate that they
intend to achieve a given bit depth and
in the case of srgb it is intended to
use with 8 bit and sign
integral now the other relevant bit for
our Basics is color
Management in the analog era there was a
single color space this color space was
standardizing recommendations from the
international telecommunication
Union for instance the current version
of the standard which is bt79 from 2015
standardized the cor space that is
relevant for digital standard definition
TV
content of course because it couldn't be
any other way each broadcasting standard
from from each country gives rise to
several variation of this broadcasting
color space and we will cover this
examples
later however in the digital era we have
a different tool this tool is called
color Management
Systems today's color Management Systems
followed a framework that is prescribed
by another standards organization called
the international color
Consortium this framework is called open
and loop color
management and all calculations are done
within what we call a profile connection
space this space is an intermediate
device independent color
space that means that it is very unusual
to see output or input in this
particular color space rather we make
all the conversions to and from
it that also means that primaries and
illum are not defined in the source or
the destination cor space rather they
are defined in this profile connection
space finally whenever we need to take
color data from or to a device it means
that we need to transform the data from
the device to the
PCS and from the PCS to the destination
color space please know the inverted
directions in this
transformation this figure illustrates
the open loop color
management there are many input devices
in this case cameras and you have
different types of of output devices
such as a computer screen a mobile phone
and a laser and an inch
printer in all these cases the device
will tell you how to transform from its
color space to the PCS and the output
device will tell you how to transform
from the PCS to its own output color
space now to actually do color
management as described by the ICC you
need four key
components first you need a common
profile connection
space then you need a cage management
module which is usually a software
library that performs all the colum
conversions it comes usually embedded on
your operating system though if you need
any particular functionality or you need
a customized offering there are
offerings available from other
vendors third you need a device profile
which contain the data to transform
between the PCS and your device color
space this also means that our space are
cont within the device
profiles and finally you have rendering
intents this is because there are
usually no exact matches between h two
given spaces and this gives rise to what
we call out of G
colors it is the responsibility of the
color management system to particularly
account for this and given a color that
is out of gamut and return a reasonable
answer in the destination color
space now this is as much as we are
going to cover as regards the ICC color
management architecture if you're
interested in more I spoke about it at
length in debox in
2022 I'm leaving the link in the
description as well as in the QR code
here with this I now conclude the first
section so let's move on to the second
one in which I will ask the question
what are these
spaces to explain this we need to start
thinking about the reason for existing
of these color
spaces this spaces are made to digitally
encode color signals for transmission
this means either TV nowadays streaming
or storage
Etc tombs there is a few requirements
that are good H color space needs to
cover first of all it needs to retain
color
balance that is to say that a color
drift between different cameras is bad
also color drift between different
screens is
bad second objective is to preserve the
lightness information this is because
our eyes are more sensed to light rather
than to color
itself and the objective is to achieve
efficiency when transmitting Coral
signals for instance if we think of an
common screen cap 1080p 8bit RGB which
is what most color screens uses
nowadays if we try to store that
completely
uncompressed we need as much as 170
megabytes to store all that color
data the question that we need to ask
ourselves is how how much of this space
or this bandwidth if we are streaming or
transmitting is actually
needed to clarify what I said earlier a
bit we need to think about two terms
light and color the whole purpose of a
color space is to represent Faithfully
the response of the ey to light stimuli
to achieve this this lumac chroma color
spaces split color information into two
kinds of
component the first one is called Luma
which is also known Cally as gray scale
lightness Etc and represent the
Luminosity information of the
image if you have seen monochrome images
this is exactly what Luma
is and the second component is chroma
that represents actually the color
itself now for an actual more precise
definition of luma or luminance if you
have heard this term previously we need
to keep in mind that image transmission
involves a process that is called gamma
correction in the cor space standards
however these are not actually called
gamma correction that's a misnomer from
the analoga they're actually called opo
electric transfer function this one is
applied when turning the for instance
the input of the camera into a
file and there is a counterpart called
Electro opot transfer function this
function takes the that is stored in the
color image and turns it into the actual
response that our eyes will
see these two different functions are
necessary because the both the image
sensors and the Display Devices have
something called
nonlinearity nonlinearity means that
given two consecutive levels of gray our
eyes perceive them more differently that
they are actually in real
life so let's see actual example of how
gamma correction Works in real life we
have on the left any input devic it in
this case different types of
cameras if this cameras output srgb the
transfer function will turn that into
linear
RGB which is then transformed into the
PCS by the color management
system another transformation takes it
from the PCS back to linear RGB
and finally the auto electronical
transfer function compar it back into
srgb and so we can see it in for example
a screen a mobile phone a TV or maybe
stor it into a
DVD previously I mixed a couple of terms
Luma versus luminance you may have also
heard about chroma versus chrominance or
this
interchangeable Luma and chroma
officially mean gamma correct Channel
you will see for instance in a standard
definition a little TI attached to the
variable name this is usually apply to
the Luminosity Channel only though in
very formal context they will apply to
all the
channels for instance if I write I till
the cbcr which is one of the examples
that we will cover later this mean that
the color space will transform it from
gamma corrected
RGB and there is another definition
which is luminance versus chrominance
these two terms mean linear or
uncorrected
channels for instance if we have I
without the CBC this means that the cor
space is based on linear RGB and this
would result in very worh colors if you
try to use it with gamma corrected
RGB at this rate you may wonder why all
this bagage
this whole bit of theory is necessary
because preserving color balance as I
said earlier is
essential any error every mismatch that
the color management system introduces
in Luma is very noticeable to the naked
eye however if we make a mistake with
chroma the ey may not see
it this is a key Insight that enables
codex to both drop spash resolution and
then loely compress the color dat
there's a huge benefit in this with this
change less bandwidth or storage is
required we achieve faster transmission
rates and for instance the same TP frame
that we had ear that was 170
megabytes if we turn it into a JPG file
we can squee it to less than a
megabyte there are several examples of
this for instance GPH h264 the video
standard H ABC the and new standard bp9
ab1 the both codes that YouTube and
twitch use now etc etc all this compress
the data a lot and make it far more
affordable to be transmitted on streamed
online this whole H SP res solution
dropping process is called actually
chroma sub
sampling in this slide we see different
types of chromazone sampling
in the First Column we see a 4x2 a Luma
sample in the second column we have
different types of chromas sub sampling
for instance in the first one we see
four pixels of
chroma in the second the chroma column
first we have the 4x2
block then we doop two in the horizontal
Direction the other example we drop two
but in the vertical Direction
and finally we drop two and two on each
Direction This give rise to different
chromat subsampling standards called
444 422 420 and 411
respectively now finally we reach the
mid of the stock that is the examples
section lumac chroma color spaces are
special in the sense that they are
usually transformation of existing color
spaces this means that unlike the cor
spaces that we see in the theory these
ones require as a definition the base
cor
space then the transformation function
which is usually though not always see
why later a linear transformation The
Source
space and finally some cor spaces May
prescribe transfer functions for the
their operation and this should be
explicitly made
available so let's begin with the first
example which is the color space called
y
IQ this color space was created in 1953
for one of the first analog broadcasting
standards which was
ntsc this makes it as far as I believe
the oldest lumac chroma space in the
Imaging
field it was a standard in two different
itu standards the first is called BT 470
from
1998 which was super seed for
compatibility Reasons by BT
1700 which is from
2005 in this color space we turn gamma
collected RGB pixels into y IQ signal
values where y Prime is the Luma signal
this is because as we said earlier and I
are most sensitive to the G part of the
RGB pixel and G is a major contributor
to the Luminous
value and R and B have smaller
contributions then I and you go together
form the other part of the signal which
is called the chroma signal these are
complimentary color difference signals
on the right we have one example that
was taking from Wikipedia showing the
whole composite image and the different
parts of the color
signal a quick overview now of how the
yiq color space standard is
Define first this color space was
designed to be used for standard
definition Aller
[Music]
Transmissions second it uses a very old
illuminum called illuminant C which was
defined the original set which was the
made in 1931
unfortunately this color space was
quickly made obsoleted with invasion of
another broadcasting standard called
pal this was unfortunate because at the
time the ntsc broadcasting stand a very
expensive analog cry to stabilize the
signal and here we have the set of
equations that transform the RGB gamma
collected signal into
yiq as with most color spaces that were
made for broadcasting standards y IQ
takes and returns a gamma corrected RGB
signal and also defines its own
transform functions which are shown
below first we have the ootf the electr
transfer function which relates the
input luminance to electrical
signal and then we have the counterpart
that Rel the electrical signal to the
output
luminance
for the second example of this stock I'm
going to talk about one of the rarest
cor spaces out there which is ID
PTR this color space entered used in
France in 1967 though there were a bit
studies done much earlier as early as
1960 and his objective was to fix ntsd
need for facing an adjustment that was
the issue that I spoke about earlier it
also Al consumes CA corrected RGB pixels
and outputs I prime dbdr signal values
on the right again we have an an
example Y is the Luma signal and dbd
andd together form the chroma
signal idbdr was
standardized together with ntsc and B in
bt47 from
1998 however due to ofen and Rarity it
didn't make it to the upgrade BT 1700 so
that was made
obsolet it also targets a d65 white
Point does beting ICBC which comes next
however unlike ibcr it targets the
domain entirely so it doesn't have any
transfer
function it assumes instead a power log
called gamma which equal equals to 2.2
for nonlinearity of the
display
now the start of the show y Prime
cbcr this color space was created in
1981 as a join project of the European
broadcasting unit and the
smpte its objective is to H create a
power an entire Digital Signal
compositing
pipeline it also consumes gam corrected
RGB pixels and TS them into y Prime CB
and CR signal values where y Prime again
is the Lum the Luma signal and CB and cl
together form the chroma
signal in this case CB and CR have a
specific meanings CB is a color
difference signal comis of blue minus
Luma and CR is the difference between
red and
Luma there is a Greek property that less
than 25% of this color space reaches to
valid RB colors according to is RIT
CO as we will see later this is a
critical property ensuring the success
of ycdc which is a bit
conely but it was standardized by the in
not one but three recommendations that
we will see
next the first version BT 601 was L
updated in
2011 it targets a standard definition
transmissions and was designed strictly
for compatibility with Legacy that is to
say monop Chrome and
tsvs it also targets a d65 y point and
has the very same gamma correction as
yq and here of course are the equations
converting from RB to
ycbcr the second version was
standardized as PT709 last updated in
2015 it is a revised version that was
designed to Target a high definition
Transmissions
this version drops Legacy compatibility
in exchange for a more accurate
luminance response that models the
response of the
eye however it still targets a d65 w
point and it still uses the same gam cor
as
bt601 as you can see now the equ change
quite a bit compared to
bt601 finally we have the BT 2020
version which was last updated in
2015 this is a revised version that
powers 4K definition and HDR
Transmissions it upgrades both the
transformation matrix and the transform
function but maintains thisi way
point as you can see the the equations
have changed quite a bit compared to the
original
versions and it also makes its own
definition as CS gamma correction
BT 2020 unlike previous standards Define
both corrected and uncorrected version
for the sake of a consistency in this
stock we will cover only the gamma
corrected ICBC version but since we
don't have a 8bit dep I will cover
instead the 10B T which isign for
HDR Again thef release the input
luminance to the electrical signal in
this case only the one of the
coefficients
change and the eotf F relates electrical
signal to the opp luminus in this case
it didn't have any
changes now for one of the more
undocumented color spaces we have Sy ycc
which was a standardized in 1999 as part
of srgb this is part of an ESO
standard this color space is used by GPH
to convert the color data prior to the
compression step that's called the C
it also consumes gamma corrected RGB and
turns it into
ycbcr however unlike most of the I
standards it uses its own primaries
which are based on the srgb primaries
not in CSC it is or9 or
2020 this also means that it allows H to
Target the extended gamut of uh X
ycc that is to say if you have colors
that don't fit within srgb but you can
capture them with your device you can
write them directly into Sy
ycc we enter now the FY digital ERA with
the first color space
ycg which was standardized as a
recommendation h. 273 in
2016 this one turns gamma cor the RGB
pixels into y CG and Co signal
values where CG and Co unlike previous
color spes inste represent orange and
green chroma not chromance as the
Wikipedia page
suggests we can see the difference in
the right where we have a very clearly
separated Luma and the other two planes
representing CG and Co in that
order YC GCO as a said is standardized
as R recommendation h. 273 this standard
is designed to provide a single point
for all transfer function partitioner
could probably use and its main property
is that it was designed for integer
mat it design makes it so that for
instance if you have 8bit srgb and you
need to convert it to YC GCO you only
need 10 bits to preserve the full n bit
RGB range unlike previous color
spaces and here you can see the Matrix
transformation between gamma corrected
RGB and ycg NCO you can see that the
Matrix is far more simplify compared to
the previous
equations and for the final example of
this stock the newest color space I
could find is
ICP which is a color space designed by
Doby L charges in
2016 this space turns a gamma corrected
RGB pixels into y CT and CP signal
values please note that there is an
Optive fully linear RGB
version in this case I is the intensity
or Luma Channel and CT and CP together
from the chroma
channel in this case CT is called chroma
Tian and represents the yellow blue hue
of the color image and CP is Chrome
proton it represents the red green part
of the
image this space was di a part of the E2
recommendation bt21 100 which was
published in
2018 ICP is the first lumaca color space
that it is not a linear transformation
of a previous color
space this space unlike all others
requires for the first time a complete
color management transformation Pipeline
and it also requires the B2200 transfer
function for a high dynamic range
this means that to go from RGB gamma
corrected to iccp we need to go
first from gamma corrected RB to linear
RGB then we need to go to a score space
called
LMS and there we can apply the
transformation matrix to go into
ycp unfortunately to explain how LMS
work and why iccp is so important I will
need to go into much bigger depth as
regard color space we need to go into
the whole technical theory behind how we
perceive colors and this would turn into
a whole different
talk and so we arrive to the last
section the conclusions of this talk we
have seen quite a few color spaces they
were mostly designed for analog
broadcasting but thanks to the Advent of
computers and phones and so on now they
are presented on all everyday multimedia
applications and we covered a basic
primary of color
spaces we so cover the basics of color
management why are lumac chroma color
spaces so essential and finally the
examples that are in W use
today as I said earlier we didn't cover
the full background behind color spaces
for instance we didn't cover Concepts
such as Tracy LC response functions and
so
on we also didn't cover what's needed
for 4K and HDR transmissions for
instance why ICBC is so surpassed by
iccp in digital
workflows this is because the technical
Concepts behind iccp are so complicated
that would make this whole talk a at a
very different technical level than just
an
introduction but if you are feeling so
inclined to read a bit more about H
Color spaces I the extra references at
the end of the slides provided with this
talk so that's all for today thank you
all for watching if you have any
questions we have a Q&A section next or
if you want you can write an email or
drop a line of Matrix and I will answer
as soon as I
can thank you all for watching see you
soon
welcome back everyone um thank you for
for the amazing talk talk Amy uh we've
got a few questions so first of all
ycbcr is it still in wi use
today yeah H I know for a fact that some
applications Like Satellites cannot put
diff images in that format but they are
really extremely extremely rare I could
never get a sample of one of them and to
the best of my knowledge there is only
just the one application that can
support and compressing viewing those
images and for that is for the of please
don't ask about ICBC gbg compressive
because those are the most C type of
file format and they best left in the
history how difficult would it be to
explain iccp compared to
ycbcr well H ICP is not a linear
transformation over RGB like
every lumac chroma color space that came
before it is actually a transformation
over how eyes perceive the color and to
explain that I need to go to the very
basics of color spaces in particular why
we need three primaries and not four and
not two that's called dromy Theory and I
will also need to explain one is
standard on its own that c standard
Observer functions those are the ones
that describe how we see colors and
numbers to it so that will be a whole
different
talk so why didn't you mention color
spaces like OK lab cie Lab lch or
XYZ I didn't have time unfortunately
here in Argentina we had a massive storm
some two weeks ago we lost power for a
whole weekend the Uplink was also cut
until the
Friday and last Friday and I needed to
hand in a big very project before we
went all at work for the Christmas break
so if anyone wants to hear more about it
please H write some lines on my Matrix
or or send me an email and I make sure
to put together a part two of the stock
that sounds amazing thank you so what
else are you working
on uh at present I'm working uh as in
the assembly and Sim region I just
finished a per project for work
I implemented a ABX acceleration for G
streamer that was a fourth month uh long
project and which I ended up H trying to
finish by looking into the mro code
implementation of
amdm it was really something that was
far more complicated that I that I had
imagine and I hope that we can get to
share that with you very very soon that
honestly sounds like a really
interesting project I think i' enjoy
watching you talk about that so anything
you want to work on in the
future well H there has been a kind of
an achievement to be unlocked for me
that is working in HDR that's high
dynamic
range uh so to put it in very simple
words imagine that our screens can put a
certain amount of light and if you have
a screen that supports
HDR you can literally l put like three
or four times the the the amount of
light and also cover a much quier range
of color but unfortunately in order to
do that you need a whole lot of
equipment it cannot be simulated with
normal screens and that's of course
really really expensive but I hope to
get to work on that in next
year honestly that's a shame that the
equipment is uh not easily
affordable um thank you very much for
your talk it was extremely interesting
and it was uh an excellent uh addition
and the F review Into The Magic of color
um thank you I've been told not to ask
you about the car so I will not do so uh
one last thing people in in the chat
have been meowing so
meow
meow thanks again for your talk um thank
you yes
sir