The second layer.

What the second layer reads

Each second-layer kernel is 3×3×8: it reads a 3×3 window of all 8 pooled first-layer maps at once, as chapter 4’s kernels with depth did. So a second-layer channel can’t see the image directly. It sees how strongly each first-layer channel responded, nearby.

It also sees more of the image. Chapter 6’s rule gives the receptive field after each layer, $r_{\text{new}} = r + (k - 1)\,j$ with $j_{\text{new}} = j\,s$, starting from $r = 1$ and $j = 1$ at the input:

Layer	Field	Jump
5×5 conv, stride 1	1 + (5 − 1) × 1 = 5	1
2×2 max pool, stride 2	5 + (2 − 1) × 1 = 6	2
3×3 conv, stride 1	6 + (3 − 1) × 2 = 10	2
2×2 max pool, stride 2	10 + (2 − 1) × 2 = 12	4

A first-layer unit sees 5×5 pixels. A second-layer unit sees 10×10, more than a third of the 28×28 image’s width. After the last pooling, each of the values the dense layer reads covers 12×12.

The same probe, one layer up

Figure 9.3 runs the same turning line through the network and records the second layer’s 16 channels, after ReLU.

The curves look different. Many second-layer curves have sharper peaks, and some drop all the way to 0 over a range of angles. Channel 3 gives 0 for 23 of the 36 angles. Their selectivities run from 0.56 to 1.00, and half of the 16 reach 1.00: there’s some angle at which they’re completely silent. Compare the first layer, whose highest was 0.70.

How does a layer of broadly tuned inputs give narrowly tuned outputs? A second-layer kernel has a weight for every first-layer map, and some of those weights are negative. So a channel can add up the maps that respond to one range of angles and subtract the maps that respond to another. Then ReLU sets everything below 0 to 0. A line at the wrong angle drives the sum below 0 at every position, so the whole map comes out 0. Being silent is something ReLU makes possible: without it, the map would just be negative.

One channel stands out the other way. Channel 7’s tuning curve barely leaves 0: its strongest response to any line is 0.06. Yet on real digits from the test set it’s the opposite of quiet. Its strongest response across them, 1.41, is the largest of any second-layer channel. Whatever channel 7 responds to, it isn’t a straight stroke. Draw some curves and corners in figure 9.1 and watch its map, the eighth of the second layer’s sixteen.

What this does and doesn’t show

The pattern, broad tuning in the first layer and narrow tuning in the second, is a measured fact about this network. It matches a common account of CNNs: early layers respond to simple local features, and later layers combine them into more specific ones, each seen over a bigger window.

It doesn’t say what each channel is for. A tuning curve shows how a channel responds to one kind of input we chose. Lines are easy to reason about, but digits aren’t made of straight lines alone, and a channel that ignores lines, like channel 7, may matter as much as one that’s sharply tuned to them.