Color images.

A screen makes every color by mixing three lights: red, green and blue. So a color image stores three numbers per pixel: how much red, how much green and how much blue. Each is between 0 and 1, as with gray. (0, 0, 0) is black, (1, 1, 1) is white, and (1, 1, 0), full red and full green with no blue, is yellow.

Three grids

Group those numbers by color instead of by pixel, and the image comes apart into three grids, each the size of the image. A channel is one of those grids: one number per pixel, all measuring the same thing. A color image has three channels, red, green and blue, usually written R, G and B. A grayscale image is the special case with one channel.

So a 32×32 color image is $3 \times 32 \times 32 = 3{,}072$ numbers. Its size is written with three numbers, height × width × channels: 32×32×3.

Figure 4.1 shows the duck’s three channels in gray. Each one is an ordinary grayscale image, and you read it the same way: white means a lot of that color, black means none.

• The duck’s body is yellow, (1.00, 0.84, 0.20). It’s white in the red channel, light in green, and dark in blue.
• The water is blue, (0.19, 0.46, 0.84), so it’s dark in red and bright in blue.
• The sky is a pale blue, (0.77, 0.89, 0.94). It’s light in all three channels, which is what makes it pale.

The red channel can’t tell the duck from the sky: the duck is white there, and the sky, at 0.77, is nearly as light. The blue channel does better. The duck is dark in it, around 0.2, and the sky and water are bright, 0.84 and up. But what really marks a yellow duck is a combination: lots of red and little blue. Red minus blue is 0.80 on the duck’s body, −0.17 on the sky and −0.65 on the water, a wider gap than either channel gives on its own. The next module builds a kernel that computes it.

Switch the figure to tinted to draw each channel in its own color.

Adding the light back up

The three channels are separate grids, but on a screen they shine from the same spot. Figure 4.2 draws each channel as a plane of its own colored light, red, green or blue, and stacks the planes in space. Wherever planes overlap on screen, their light adds. Drag to turn the stack.

Press “Stack, from above”. The planes sit on top of each other, every pixel’s red, green and blue line up, and the light adds back to the color duck. Red light plus green light, with little blue, is yellow: that’s the body. All three at nearly full strength is the pale sky.

Now pull the planes apart and turn the stack. The three copies of the duck slide out of line, and the overlap breaks up. Where only the red plane covers a spot, you see red; where red and green overlap without blue, yellow; where all three line up, the true color. Then press “Apart, from above”. Looking straight down, the planes line up again however far apart they are, and the color duck comes back.

Switch channels off to see what the others add up to. Without blue, the sky turns a yellowish green, because its pale color is red and green light with blue on top. Without red, the duck turns green. Each channel holds one part of every pixel’s color, and the image needs all three.

Channels are just stacked maps

Nothing in the definition of a channel says it has to be a color. It’s any grid of numbers the size of the image, stacked with others. That matters later in this chapter: a convolution layer’s output is also a stack of channels, and those channels don’t mean red, green or blue. They mean whatever their kernels compute.