Due date : Tuesday, March 6

We want to study the behavior of training algorithms on the computation of the n-bit parity function ( see last page of the paper " Learning internal representations by error propagation " by Rumelhart, Hinton and Williams that was handed out in class) . This problem has an exact solution, but the difficulty in training increases rapidly as the number of bits n increases.

Consider a Feed-forward, multilayer Neural Network as described in the paper ( n input, n hidden neurons, 1 output neuron ). Use the Neural Net toolbox in Matlab to implement and train this network.

The efficiency ( success, number of iterations required ) of the training algorithm depends on the particular algorithm, and it is also a function of the initial conditions. Since the latter are picked at random, the number of iterations required, and the outcome ( success or failure in training the network )

are also random variables. We want to study the dependence of the average number of iterations required, as well as the percentage of successes in training as a function of the type of algorithm used, and the number of bits n in the parity function.

For each algorithm, and for each n, try training your network for a set of different, randomly chosen, initial conditions ( say 20 cases ). Compute the percentage of times that the training was successful ( all input vectors must yield the correct output ) within a fixed number of iterations allowed ( maximum number of iterations might depend on the algorithm you choose, and the number of bits ), and the average number of iterations required for the successful cases.

Compare the different algorithms, and the scaling with respect to the number of bits n.

Here are the RULES for assignments :

I.- The due date will be strictly enforced, since I will describe the solutions in class the day the assignment is due. Any late assignment will be penalized by 50% the following class, and it will not be accepted at all afterwards.

II.- I expect a report which includes listing of your code, your comments and answers to my questions and any relevant plots. Just writing a program is not enough, actually it is the shortest and easiest part of your assignment. Running your program and drawing the correct conclusions, understanding your results and describing and reporting them in a short and accurate form are the main objectives.

The TOTAL LENGTH of your report is limited to 5 pages ( excluding the program listing ) . ANYTHING longer I WILL NOT ACCEPT .

III.- Your work is INDIVIDUAL. Although you can discuss the assignment with your classmates, you are expected to write your own program, do your own thinking, write your own report. ANY CHEATING WILL BE SEVERELY PUNISHED.