Specification of a new MLP architecture |
Input nodes:
Each node in the input layer represents an input feature from the data set.
Output nodes:
Each node in the output layer represents an output feature expected from the network.
Bias:
You can check this box to have bias included.
Hidden layers:
An MLP has at least one hidden (intermediate) layer, but you can specify any number (n) of layers.
The input neurons are fully connected to the hidden layer neurons, which are in turn fully connected to the output neurons.
Number of nodes:
Each hidden layer has a number of hidden nodes (neurons) which can be specified here.
If you have specified more than one hidden layer, then you are able to specify different values of nodes for each of these layers.
Choosing the number of hidden nodes |
The selection of an adequate architecture is one of the most difficult problems when dealing with neural networks.
There are many suggestions about how to choose the number h of hidden nodes in an MLP. For example, the minimum number should be h>=(p-1)/(n+2), where p is the number of training examples and n is the number of inputs of the network.1
If the number of hidden nodes of a network is too low the unknown function2 can not be learned (underfitting). If the number is too high the network learns the noise3 contained in the data set (overfitting) and it loses its ability to generalise.
Thus, the aim of building a neural network model should be to select an architecture that is just big enough to capture the unknown function. The best method for this is by selection of an architecture that produces the smallest root mean squared error over the test set.
[1] N.Kasabov. Neural Networks, Fuzy Systems and Knowledge Engineering, MIT Press, USA, 1996, Page 278.
[2] Unknown function: [3] Noise: Once you have specified your network architecture press the
Create button, or select Create Network from the
Network menu. This creates a network file with randomly
initialised connection weights. To retain the network for later
usage you will need to save it to a file.
Although you do not need to save the network to train and recall
it, it is better to do so. This allows you to compare the results
of different training parameters, and prevents loss of work if
something goes wrong. To save the network, select Save As
from the Network menu. Here you are able to choose what
directory you want to save your network in, and specify the name of
the file. The convention for the file extension of network files
is *.wgt. The only restriction is that the path cannot
contain a ). You can press the View Network File button to view your
network, or alternatively select View Network from the
Network menu. Note: You need to associate files of type
*.wgt with a text editor in Windows Explorer in order to view the
files through the FuzzyCOPE GUI. Below is an example of a
network file.
[FormatVersion = 1.0] /* This is a standard version number. Any value larger than 1.0 will not work*/
[NeuralNetwork]
[NetworkInformation] /*This section of information sets out the layout of the network*/
/*Details for each of the neuron layers*/
[Layer = 1] /*Layer 1 is always the input layer*/
[Layer = 2]
[Layer = 3] /*the output layer*/
/*Details for each of the connection layers*/
[ConnectionLayer]
[ConnectionLayer]
[ConnectionLayer]
[~NeuralNetwork]
It is possible to load an existing network by selecting Load
Network from the Network menu. The file can have any
extension, but the convention is for network files to have the
extension *.wgt. The only restriction is that the path
cannot contain a ).
Creating the network
Saving the network file
Viewing the network
[NetworkType = MLP]/*This shows the type of network being used - MLP / FuNN / Kohonen*/
[NumberOfLayers = 4] /*The number of layers includes the bias layer, even if it is not used*/
[InputLayer = 1] /*The input layer is always layer 1*/
[OutputLayer = 3] /*The output layer is the last layer*/
[NumberOfInputs = 4]
[NumberOfOutputs = 3]
[BiasLayer = 0] /*The bias layer is always layer 0. Note that numbering of layers and neurons starts at 0*/
[~NetworkInformation]
[Layer = 0] /*Layer 0 is always the bias layer and is included even if it is not used*/
[LayerType = Simple] /*The type of the bias layer is always simple*/
[NumberOfNeurons = 1]
[All ActivationFunction=bias ]
[~Layer]
[LayerType = Input]
[NumberOfNeurons = 4]
[All ActivationFunction=linear ]
[~Layer]
[LayerType = Gain]
[NumberOfNeurons = 3]
[All ActivationFunction=sigmoid ]
[All Gain=1.000000000000000 ]
[~Layer]
[LayerType = Gain]
[NumberOfNeurons = 3]
[All ActivationFunction=sigmoid ]
[All Gain=1.000000000000000 ]
[~Layer]
[ConnectionLayer] /*The connections in this layer are*/
[LayerFrom = 0] /*From neuron layer 0*/
[LayerTo = 2] /*To neuron layer 2*/
[ConnectionType = Pruneable] /*Connection type can be Pruneable / Unprunable*/
[Attributes Pruneable Freezable ] /*Attributes can be Pruneable / Unprunable, Freezable / Unfreezable*/
[Matrix Weights TrainableInfo ]
/*Below is the matrix of connection weights for this connection layer*/
/*In this case there is only one neuron in layer 0 and it is connected to each of the neurons in layer 2*/
[~Matrix]
/* To->
0
1
2
From */ 0.926877651295511
T
0.776543473616749
T
0.161595507675405
T
/* 0 */
[~ConnectionLayer]
[LayerFrom = 0]
[LayerTo = 3]
[ConnectionType = Pruneable]
[Attributes Pruneable Freezable ]
[Matrix Weights TrainableInfo ]
[~Matrix]
/* To->
0
1
2
From */ 0.777275917844172
T
0.19296853541673
T
-0.496505630664998
T
/* 0 */
[~ConnectionLayer]
[LayerFrom = 1]
[LayerTo = 2]
[ConnectionType = Pruneable]
[Attributes Pruneable Freezable ]
[Matrix Weights TrainableInfo ]
[~Matrix]
/* To->
0
1
2
From */ -0.701834162419507
T
0.019562364574114
T
-0.408062990203558
T
/* 0 */ 0.460982085634938
T
0.117099520859401
T
0.927060762352367
T
/* 1 */ 0.883297219763787
T
0.463118381298257
T
0.840144047364727
T
/* 2 */ -0.625171666615802
T
-0.688222907193213
T
-0.17856379894406
T
/* 3 */
[~ConnectionLayer]
[LayerFrom = 2]
[LayerTo = 3]
[ConnectionType = Pruneable]
[Attributes Pruneable Freezable ]
[Matrix Weights TrainableInfo ]
[~Matrix]
/* To->
0
1
2
From */ -0.802117984557634
T
-0.578722495193335
T
-0.686391796624653
T
/* 0 */ 0.592516861476486
T
0.062898648030032
T
0.067659535508285
T
/* 1 */ 0.426313058870205
T
0.17581713309122
T
0.788995025482955
T
/* 2 */
[~ConnectionLayer]
Opening an existing network
This page is maintained by Melanie
Middlemiss mmiddlemiss@infoscience.otago.ac.nz
Last modified on: 4/2/98.