A neural network on the GPU - Concluded

OK, it's "finished". Here's a summary of the features:

- Two-layer, feed-forward neural network for supervised
land cover/land use classification of multispectral imagery,
fully integrated into ENVI.

- Makes use of the cross entropy cost function and softmax
outputs to model posterior class membership probabilities.

- Trained with the scaled conjugate gradient algorithm,
guaranteed to converge monotonically to a (local) minimum
in the cost function, see the Figure. The algorithm is
described in detail in Appendix B of my book.

- Apart from the number of neurons in the first (hidden) layer,
there are no adjustable parameters whatsoever.

- Uses the R-operator method (Bishop 1995, Neural
Networks for Pattern Recognition) to evaluate the Hessian
(matrix of second derivatives of the cost function) efficiently.

- All matrix operations are carried out on the GPU using
GPULib IDL bindings to CUDA. CUDA is giving me a speedup
factor of around 3-4 on my system (see earlier blogs)
relative to the CPU version.

- One heck of a lot faster than ENVI's built-in network
classifier. Also much faster that ENVI's support vector machine,
with comparable accuracy.

- You can get the code here.