Fully Connected Layer

The result coming out of the Convolution/Pooling layer combination is still in the form of features. To classify or come to a conclusion we need to take into account all the data or feature that we have collected so far and check all the possible combination. This is the job of the Fully Connected layer, which is basically our regular Neural Network that we learn prior to CNN, where all the nodes are connected to nodes from the previous layer.

As shown in the picture below, the output of the last pooling layer is serialized before it is fed into a fully connected layer. In this example only one fully connected layer is applied. Since there are 8 neurons in the output of the fully connected layer, this example architecture can be applied for a classification into 8 classes. In this case the output is usually processed by a softmax-activation function, which is not depicted in the image below.

Note that the fully connected layer only accept 1 Dimensional data. To convert our 3D data to 1D, we can use the function flatten in Python. This essentially arranges our 3D volume into a 1D vector.

Softmax

A softmax operation serves a key purpose: making sure the CNN outputs sum to 1. Because of this, softmax operations are useful to scale model outputs into probabilities with a range from 0 to 1.

This topic has been covered earlier, so if you need to refresh, then please visit the appropriate course.