Neural Networks (NN) can be used to find features in digital
images. We can imagine an ideal network
that takes a sub image as input and outputs the discrete yes or no
whether or not the feature is present in the presented sub image
. The window can be tried at every
position in the image. The highest activation of our NN will represent
a high degree of certainty. This method has been described in
[Per95].
This can be achieved by training the network with the desired pattern to recognize. It is also important to include many negative examples in the training set, otherwise the network would be very optimistic and output ``yes'' at every place in the image. Choosing the training set can be very tricky and it is the crucial part of our algorithm. On one side, we would like to have a low first order error, i.e. if the feature is present, we would like a high activation. This means that the NN needs to generalize well and not to be too biased towards the training data, which is also called Overfitting. On the other hand, we also want to make few second order errors, i.e. have a high output activation if there really is no feature. That's the reason to require also good negative examples. The training set therefore has a big importance on the quality of the final network.
Our task consists of finding hyperbolae in the image, because a buried object is represented by a hyperbola on the GPR image, as previously stated.
We used a 2 layer, backpropagation neural network with a sigmoid activation function. This function was already discussed before and is printed in Figure 12. The input layer consists of 120 inputs, as the window to find the hyperbolae is 40 by 30 pixels large. One output neuron collects the activation of the 11 hidden neurons.
Building a training set was not easy at all, as there were very few
GPR images containing mines. To train a reliable network, 
50
positive examples and 150 negative examples of good quality
are needed. Therefore, we needed to generate
our own training data.
We want to find hyperbolae, so we created them with the Matlab script
hyperbola.m. The function takes many arguments specifying the
opening and width of the produced hyperbolae, plus additional parameters
setting the image size, range of pixels, ...etc. One example of
such a generated hyperbola is in Figure 17. The Matlab
code hyperbola( 1, 1, 0.5, 2, -1, 1 ) generated this specific
curve.
Figure 17: Artificially created Hyperbola
The negative examples were generated by taking 150 random images, having the same range and mean value as the real GPR images. This is not really optimal, but having very few images, we did not have a choice. We used a fast backpropagation algorithm to train our NN. This algorithm changes the learning rate and the inertia parameter during the training phase, to an optimal value. Using this method, our NN converged much faster. Training took between half an hour and 1 hour on a Sparc 20 with the Matlab NN Toolbox. Once we finished training, we tested the resulting net on each pixel of Sand3. The resulting activation diagram can be seen in Figure 18 and the activation surface is plotted in Figure 19.
Figure 18: Activation pattern of the
NN applied to Sand3
Figure 19: Activation surface of the
NN applied to Sand3
Ideally, we would expect an activation pattern that is very low on most parts of the image, but with some peaks in the presence of any hyperbola like structure. The resulting activation is very promising. Over most parts of the image, the activation is almost zero, but in the parts that contain the structure there is some activation. The activation is also present in areas containing not really an entire hyperbola. This is clear, as we did not add real negative examples. All the cases of ground noise, half a hyperbola or others, need to be added as negative examples, which would further improve the accuracy.
To ease the task of our network, we changed the image
representation. Hyperbolae can be represented by directions or a
gradient flow. In the intensity image, the
neurons need to create some internal method to compute the gradient
flow to predict hyperbolae. It might be a lot easier for our NN to
present it a gradient flow diagram. We used a linear symmetry
algorithm to compute the local flow of our images. Training was done
using the flow of our self generated hyperbolae images. Unfortunately,
the flow at each point is represented by 2 vectors, 
and 
, which doubles our input size. The number of input weights is also
doubled, and the training time becomes much longer, about 1 hour on a
Sparc 20.
The results in Figures 21, 22 and 23 look promising. In Figure 22 we can see that the activations have a Gaussian-like shape. This is a good sign for our training method. An activation similar to a Gaussian shows that the NN is not too general and on the other side not overfitted. It is possible to use this method to decide when to stop training.
Figure 20: Simulated
Hyperbola in Quiver plot
Figure 21: Activation pattern of the
NN applied to Sand3
Figure 22: Activation surface of the
NN applied to Sand3
Figure 23: Activation surface of the
NN applied to Sand3