Welcome to mini_net’s documentation!

Author: Cole Howard
Email: uglyboxer@gmail.com

network.py is a basic implementation of a one layer linear neural network, to examine an implementation of backpropagation. It is based on the basic model of the Perceptron. Information on that can be found at: https://en.wikipedia.org/wiki/Perceptron

The intent of this specific project is to alter the Perceptron’s

decision function to a logistic function and add a “backpropagation” step at the end of each vector’s pass through the neuron.

There are several methods included that are currently passed, as they are plans to make it extensible as possible as a tool for exploring neural nets under more general circumstances.

Dependencies:

numpy.dot() : for a fast implementation of the dot product of two vectors sklearn.datasets : (optional) for running this as a script on the

scikit-learn digits dataset
neuron : the class definition of an individual neuron, also included in
mini_net
Usage:

It is currently set up to run a training set of input (along with the associated answers) and a set of similar, but distinct, input (without) the answers, and have the machine guess an answer to each of those cases based on information it gathered during the training run.

To import the network for testing on other data:

download the package mini_net, then include in your script:

from network import Network

To execute as is, from the command line, while in the linear_neuron/mini_net/ directory, input:

$ python3 network.py

This will pull the learning and test data from scikit-learn run both and return a success count, total count of the unseen test data, and the success rate that equals.

First output and success ratio will be based on the first set of testing vectors. The second set will represent the same for the validation set. The visualization (see below) that pops up, just close that window for the script to finish running.

Alternate data sets:
Alternate training and testing data sets can be swapped out in the first section of main() below. See those notes for specifics.
Visualization:
Pyplot is included to provide a visual representation of a member of the dataset.
class network.Network(neuron_targets, train_set, train_answers, epochs, test_set, test_answers, validation_set, validation_answers, images=None)

A Network instance will create layers of neurons for the implementa- tion of neural network.

Parameters:
  • images (list) – Corresponding images of the dataset
  • neuron_targets (list) – The possible final output values
  • vector_size (int) – Size of the individual input vectors
  • train_set (list) – Set of vectors for the learning portion
  • train_answers (list) – Correct answers that correspond to the train_set
  • epochs (int) – Number of times the learn_run will run for a given train_set
  • test_set (list) – Set of vectors, discrete from the train_set to have the machine guess against
  • test_answers (list) – Correct answers for the test_set, to compare the machine’s guesses against
  • validation_set (list) – A validation set to compare answers in a second run
  • validation_answers (list) – Answer for the above
neurons

Class Neuron – Instances of the Neuron class, one for each of possible correct answers

append_bias(vector)

Takes a list of n entries and appends a 1 for the bias

Parameters:vector (list) –
num_of_training_vectors

int – This is to adjust the size of the training set when all of the data is provided as large list. Breaking the training data into a training set, testing set, and a validation set. Picking this number is a balance between speed (lower number) and overfitting the data (a higher number)

Returns:The input vector with a one appended to the end of the list, as a bias
Return type:list
gradient_descent(vector, vector_index)

Calculates the gradient_descent

Parameters:
  • vector (list) – A single input, comprised of floats
  • vector_index (int) –
learning_rate

float – Determines how much of the error is applied to the weights in each iteration

Returns:Represents the error to be used to update the weights of the neurons. It should approximate a gradient descent in topology of the outputs
Return type:float
learn_run()

Runs an iteration through the neuron sets and adjust the weights appropriately. It then follows up with a second weight adjusment accross all neurons with an estimate of the gradient descent function

report_results(guess_list, validation=False)

Reports results of guesses on unseen set

Parameters:guess_list (list) –
run_unseen(validation=False)

Makes guesses on the unseen data, and switches over the test answers to validation set if the bool is True

For each vector in the collection, each neuron in turn will either fire or not. If a vector fires, it is collected as a possible correct guess. Not firing is collected as well, in case there an no good guesses at all. The method will choose the vector with the highest dot product, from either the fired list or the dud list.

Parameters:validation (bool) – Runs a different set of vectors through the guessing process if validation is set to True
Returns:a list of ints (the guesses for each vector)
Return type:list

Author: Cole Howard Email: uglyboxer@gmail.com neuron.py is a basic linear neuron, that can be used in a perceptron Information on that can be found at: https://en.wikipedia.org/wiki/Perceptron

It was written as a class specifically for network ()

Usage:

From any python script:

from neuron import Neuron
API:
update_weights, fires are the accessible methods usage noted in their definitions
class neuron.Neuron(vector_size, target, sample_size, answer_set)

A class model for a single neuron

Parameters:
  • vector_size (int) – Length of an input vector
  • target (int) – What the vector will associate with its weights. It will claim this is the correct answer if it fires
  • sample_size (int) – Total size of sample to be trained on
  • answer_set (list) – The list of correct answers associated with the training set
threshold

float – The tipping point at which the neuron fires (speifically in relation to the dot product of the sample vector and the weight set)

weights

list – The “storage” of the neuron. These are changed with each training case and then used to determine if new cases will cause the neuron to fire. The last entry is initialized to 1 as the weight of the bias

expected

list – Either 0’s or 1’s based on whether this neuron should for each of the vectors in the training set

guesses

list – Initialized to 0, and then updated with each training vector that comes through.

fires(vector)

Takes an input vector and decides if neuron fires or not

Parameters:vector (list) – A sample vector
Returns:
  • bool – Did it fire? True(yes) or False(no)
  • float – The dot product of the vector and weights
train_pass(vector, idx)

Passes a vector through the neuron once

Parameters:
  • vector (a list) – Training vector
  • idx (an int) – The position of the vector in the sample
Returns:

Return type:

None, always

update_weights(error, vector)

Updates the weights stored in the receptors

Parameters:
  • error (int) – The distance from the expected value of a particular training case
  • vector (list) – A sample vector
l_rate

float – A number between 0 and 1, it will modify the error to control how much each weight is adjusted. Higher numbers will train faster (but risk unresolvable oscillations), lower numbers will train slower but be more stable.

Returns:
Return type:None

Indices and tables