Table of Contents
Back

Specification of a new MLP architecture

Input nodes:
Each node in the input layer represents an input feature from the data set.

Output nodes:
Each node in the output layer represents an output feature expected from the network.

Bias:
You can check this box to have bias included.

Hidden layers:
An MLP has at least one hidden (intermediate) layer, but you can specify any number (n) of layers.
The input neurons are fully connected to the hidden layer neurons, which are in turn fully connected to the output neurons.

Number of nodes:
Each hidden layer has a number of hidden nodes (neurons) which can be specified here.
If you have specified more than one hidden layer, then you are able to specify different values of nodes for each of these layers.



Back

Choosing the number of hidden nodes

The selection of an adequate architecture is one of the most difficult problems when dealing with neural networks.

There are many suggestions about how to choose the number h of hidden nodes in an MLP. For example, the minimum number should be h>=(p-1)/(n+2), where p is the number of training examples and n is the number of inputs of the network.1

If the number of hidden nodes of a network is too low the unknown function2 can not be learned (underfitting). If the number is too high the network learns the noise3 contained in the data set (overfitting) and it loses its ability to generalise.

Thus, the aim of building a neural network model should be to select an architecture that is just big enough to capture the unknown function. The best method for this is by selection of an architecture that produces the smallest root mean squared error over the test set.

[1] N.Kasabov. Neural Networks, Fuzy Systems and Knowledge Engineering, MIT Press, USA, 1996, Page 278.

[2] Unknown function:In most cases MLP neural networks are used to find the relationship between some variable x and a variable y. The unknown relationship is denoted by the function f(x).

[3] Noise:The unknown relationship is often additionally disguised by noise, denoted by e.
y = f(x) + e



Back

Creating the network

Once you have specified your network architecture press the Create button, or select Create Network from the Network menu.

This creates a network file with randomly initialised connection weights.

To retain the network for later usage you will need to save it to a file.



Back

Saving the network file

Although you do not need to save the network to train and recall it, it is better to do so. This allows you to compare the results of different training parameters, and prevents loss of work if something goes wrong.

To save the network, select Save As from the Network menu.

Here you are able to choose what directory you want to save your network in, and specify the name of the file.

The convention for the file extension of network files is *.wgt.

The only restriction is that the path cannot contain a ).



Back

Viewing the network

You can press the View Network File button to view your network, or alternatively select View Network from the Network menu.

Note: You need to associate files of type *.wgt with a text editor in Windows Explorer in order to view the files through the FuzzyCOPE GUI.

Below is an example of a network file.

[FormatVersion = 1.0] /* This is a standard version number. Any value larger than 1.0 will not work*/

[NeuralNetwork]
[NetworkType = MLP]
/*This shows the type of network being used - MLP / FuNN / Kohonen*/

[NetworkInformation] /*This section of information sets out the layout of the network*/
[NumberOfLayers = 4] /*The number of layers includes the bias layer, even if it is not used*/
[InputLayer = 1] /*The input layer is always layer 1*/
[OutputLayer = 3] /*The output layer is the last layer*/
[NumberOfInputs = 4]
[NumberOfOutputs = 3]
[BiasLayer = 0]
/*The bias layer is always layer 0. Note that numbering of layers and neurons starts at 0*/
[~NetworkInformation]

/*Details for each of the neuron layers*/
[Layer = 0] /*Layer 0 is always the bias layer and is included even if it is not used*/
[LayerType = Simple] /*The type of the bias layer is always simple*/
[NumberOfNeurons = 1]
[All ActivationFunction=bias ]
[~Layer]

[Layer = 1] /*Layer 1 is always the input layer*/
[LayerType = Input]
[NumberOfNeurons = 4]
[All ActivationFunction=linear ]
[~Layer]

[Layer = 2]
[LayerType = Gain]
[NumberOfNeurons = 3]
[All ActivationFunction=sigmoid ]
[All Gain=1.000000000000000 ]
[~Layer]

[Layer = 3] /*the output layer*/
[LayerType = Gain]
[NumberOfNeurons = 3]
[All ActivationFunction=sigmoid ]
[All Gain=1.000000000000000 ]
[~Layer]

/*Details for each of the connection layers*/
[ConnectionLayer] /*The connections in this layer are*/
[LayerFrom = 0] /*From neuron layer 0*/
[LayerTo = 2] /*To neuron layer 2*/
[ConnectionType = Pruneable] /*Connection type can be Pruneable / Unprunable*/
[Attributes Pruneable Freezable ] /*Attributes can be Pruneable / Unprunable, Freezable / Unfreezable*/
[Matrix Weights TrainableInfo ]
/*Below is the matrix of connection weights for this connection layer*/
/*In this case there is only one neuron in layer 0 and it is connected to each of the neurons in layer 2*/
/* To-> 0 1 2 From */
0.926877651295511 T 0.776543473616749 T 0.161595507675405 T /* 0 */
[~Matrix]
[~ConnectionLayer]

[ConnectionLayer]
[LayerFrom = 0]
[LayerTo = 3]
[ConnectionType = Pruneable]
[Attributes Pruneable Freezable ]
[Matrix Weights TrainableInfo ]
/* To-> 0 1 2 From */
0.777275917844172 T 0.19296853541673 T -0.496505630664998 T /* 0 */
[~Matrix]
[~ConnectionLayer]

[ConnectionLayer]
[LayerFrom = 1]
[LayerTo = 2]
[ConnectionType = Pruneable]
[Attributes Pruneable Freezable ]
[Matrix Weights TrainableInfo ]
/* To-> 0 1 2 From */
-0.701834162419507 T 0.019562364574114 T -0.408062990203558 T /* 0 */
0.460982085634938 T 0.117099520859401 T 0.927060762352367 T /* 1 */
0.883297219763787 T 0.463118381298257 T 0.840144047364727 T /* 2 */
-0.625171666615802 T -0.688222907193213 T -0.17856379894406 T /* 3 */
[~Matrix]
[~ConnectionLayer]

[ConnectionLayer]
[LayerFrom = 2]
[LayerTo = 3]
[ConnectionType = Pruneable]
[Attributes Pruneable Freezable ]
[Matrix Weights TrainableInfo ]
/* To-> 0 1 2 From */
-0.802117984557634 T -0.578722495193335 T -0.686391796624653 T /* 0 */
0.592516861476486 T 0.062898648030032 T 0.067659535508285 T /* 1 */
0.426313058870205 T 0.17581713309122 T 0.788995025482955 T /* 2 */
[~Matrix]
[~ConnectionLayer]

[~NeuralNetwork]



Back

Opening an existing network

It is possible to load an existing network by selecting Load Network from the Network menu.

The file can have any extension, but the convention is for network files to have the extension *.wgt.

The only restriction is that the path cannot contain a ).


This page is maintained by Melanie Middlemiss mmiddlemiss@infoscience.otago.ac.nz
Last modified on: 4/2/98.