Where it fails.

The 216 mistakes

The network reads 9,784 of the 10,000 test digits right. The 216 it gets wrong aren’t spread evenly. One mix-up is far more common than any other: a 4 read as a 9, 41 times. Next come a 7 read as a 2, 20 times, and an 8 read as a 9, 12 times.

A 4 and a 9 can differ by very little. Close the top of a 4 and it’s a 9.

Figure 9.4 shows the first 16 test digits, in MNIST’s order, that the network reads wrong. Pick one to see its scores. Some are hard for a person too: test digit 445 is a 6 with a loop that’s easy to read as a 0, which is what the network says. Others look clear and are still read wrong. Test digit 571 is a plain 4, and the network calls it a 9.

How sure was it?

Training aimed every score at 1 for the right digit and 0 for the rest, so a score near 1 means the digit looks a lot like the training digits of that class. Most right answers come out that way: on the digits it reads right, the median winning score is 0.93. On its mistakes, the winning score is usually low, 0.46 at the median, and the right digit’s score is often close behind. In figure 9.4, look at how close the two are.

So a low winning score is a useful warning. But it’s only a warning, not a guarantee.

Inputs that aren’t digits

The last three inputs in figure 9.4 aren’t digits.

• Random noise, every pixel a random gray. The network says 8, with a score of 0.70. More than one in ten of the digits it reads right win with a lower score than that.
• A blank image. The network says 1, but every score is close to 0, and the winner, 0.21, is low.
• A straight line at 30°. The network says 1, narrowly, at 0.27.

The noise is the important one. The network has ten outputs, and the highest one is always its answer. There’s no output for “this isn’t a digit”, because every image it trained on was a digit. Its scores were never trained on anything else, so on noise they can be anything, including high.

Inputs unlike anything in the training data are called out-of-distribution inputs: the training data came from one kind of image, and these come from somewhere else. A network has no reliable way to know it’s been given one. Its scores were only ever checked on the kind of image it trained on.

That’s true of networks far bigger than this one. A network that labels photos has never been told what to do with an X-ray, and a network trained on scans from one hospital’s machines can do worse on another’s. Testing on held-back images, like MNIST’s test set, measures how well a network does on images like its training data. It says nothing about the rest.

The end of the book

This module ends the book. It started with a dense layer that couldn’t use the fact that neighboring pixels belong together, and built up the alternative one piece at a time: a kernel sliding over an image, stride and padding, channels, pooling, receptive fields, training by backpropagation, and upsampling. The network in this chapter uses most of them, and everything it does is multiplication, addition, ReLU and taking the largest of a few numbers.