Welcome to mini_net’s documentation!¶
- Author: Cole Howard
- Email: uglyboxer@gmail.com
network.py is a basic implementation of a one layer linear neural network, to examine an implementation of backpropagation. It is based on the basic model of the Perceptron. Information on that can be found at: https://en.wikipedia.org/wiki/Perceptron
The intent of this specific project is to alter the Perceptron’s
decision function to a logistic function and add a “backpropagation” step at the end of each vector’s pass through the neuron.
There are several methods included that are currently passed, as they are plans to make it extensible as possible as a tool for exploring neural nets under more general circumstances.
- Dependencies:
numpy.dot() : for a fast implementation of the dot product of two vectors sklearn.datasets : (optional) for running this as a script on the
scikit-learn digits dataset- neuron : the class definition of an individual neuron, also included in
- mini_net
- Usage:
It is currently set up to run a training set of input (along with the associated answers) and a set of similar, but distinct, input (without) the answers, and have the machine guess an answer to each of those cases based on information it gathered during the training run.
To import the network for testing on other data:
download the package mini_net, then include in your script:
from network import NetworkTo execute as is, from the command line, while in the linear_neuron/mini_net/ directory, input:
$ python3 network.pyThis will pull the learning and test data from scikit-learn run both and return a success count, total count of the unseen test data, and the success rate that equals.
First output and success ratio will be based on the first set of testing vectors. The second set will represent the same for the validation set. The visualization (see below) that pops up, just close that window for the script to finish running.
- Alternate data sets:
- Alternate training and testing data sets can be swapped out in the first section of main() below. See those notes for specifics.
- Visualization:
- Pyplot is included to provide a visual representation of a member of the dataset.
-
class
network.
Network
(neuron_targets, train_set, train_answers, epochs, test_set, test_answers, validation_set, validation_answers, images=None)¶ A Network instance will create layers of neurons for the implementa- tion of neural network.
Parameters: - images (list) – Corresponding images of the dataset
- neuron_targets (list) – The possible final output values
- vector_size (int) – Size of the individual input vectors
- train_set (list) – Set of vectors for the learning portion
- train_answers (list) – Correct answers that correspond to the train_set
- epochs (int) – Number of times the learn_run will run for a given train_set
- test_set (list) – Set of vectors, discrete from the train_set to have the machine guess against
- test_answers (list) – Correct answers for the test_set, to compare the machine’s guesses against
- validation_set (list) – A validation set to compare answers in a second run
- validation_answers (list) – Answer for the above
-
neurons
¶ Class Neuron – Instances of the Neuron class, one for each of possible correct answers
-
append_bias
(vector)¶ Takes a list of n entries and appends a 1 for the bias
Parameters: vector (list) – -
num_of_training_vectors
¶ int – This is to adjust the size of the training set when all of the data is provided as large list. Breaking the training data into a training set, testing set, and a validation set. Picking this number is a balance between speed (lower number) and overfitting the data (a higher number)
Returns: The input vector with a one appended to the end of the list, as a bias Return type: list -
-
gradient_descent
(vector, vector_index)¶ Calculates the gradient_descent
Parameters: - vector (list) – A single input, comprised of floats
- vector_index (int) –
-
learning_rate
¶ float – Determines how much of the error is applied to the weights in each iteration
Returns: Represents the error to be used to update the weights of the neurons. It should approximate a gradient descent in topology of the outputs Return type: float
-
learn_run
()¶ Runs an iteration through the neuron sets and adjust the weights appropriately. It then follows up with a second weight adjusment accross all neurons with an estimate of the gradient descent function
-
report_results
(guess_list, validation=False)¶ Reports results of guesses on unseen set
Parameters: guess_list (list) –
-
run_unseen
(validation=False)¶ Makes guesses on the unseen data, and switches over the test answers to validation set if the bool is True
For each vector in the collection, each neuron in turn will either fire or not. If a vector fires, it is collected as a possible correct guess. Not firing is collected as well, in case there an no good guesses at all. The method will choose the vector with the highest dot product, from either the fired list or the dud list.
Parameters: validation (bool) – Runs a different set of vectors through the guessing process if validation is set to True Returns: a list of ints (the guesses for each vector) Return type: list
Author: Cole Howard Email: uglyboxer@gmail.com neuron.py is a basic linear neuron, that can be used in a perceptron Information on that can be found at: https://en.wikipedia.org/wiki/Perceptron
It was written as a class specifically for network ()
- Usage:
From any python script:
from neuron import Neuron- API:
- update_weights, fires are the accessible methods usage noted in their definitions
-
class
neuron.
Neuron
(vector_size, target, sample_size, answer_set)¶ A class model for a single neuron
Parameters: - vector_size (int) – Length of an input vector
- target (int) – What the vector will associate with its weights. It will claim this is the correct answer if it fires
- sample_size (int) – Total size of sample to be trained on
- answer_set (list) – The list of correct answers associated with the training set
-
threshold
¶ float – The tipping point at which the neuron fires (speifically in relation to the dot product of the sample vector and the weight set)
-
weights
¶ list – The “storage” of the neuron. These are changed with each training case and then used to determine if new cases will cause the neuron to fire. The last entry is initialized to 1 as the weight of the bias
-
expected
¶ list – Either 0’s or 1’s based on whether this neuron should for each of the vectors in the training set
-
guesses
¶ list – Initialized to 0, and then updated with each training vector that comes through.
-
fires
(vector)¶ Takes an input vector and decides if neuron fires or not
Parameters: vector (list) – A sample vector Returns: - bool – Did it fire? True(yes) or False(no)
- float – The dot product of the vector and weights
-
train_pass
(vector, idx)¶ Passes a vector through the neuron once
Parameters: - vector (a list) – Training vector
- idx (an int) – The position of the vector in the sample
Returns: Return type: None, always
-
update_weights
(error, vector)¶ Updates the weights stored in the receptors
Parameters: - error (int) – The distance from the expected value of a particular training case
- vector (list) – A sample vector
-
l_rate
¶ float – A number between 0 and 1, it will modify the error to control how much each weight is adjusted. Higher numbers will train faster (but risk unresolvable oscillations), lower numbers will train slower but be more stable.
Returns: Return type: None