Machine learning for beginners - MP Neuron

Updated on: May 25, 2020 · 13 mins read
Categories: machinelearning   | datascience   | numpy |

MP neuron is the first step toward opening up the doors of artificial neural networks and artificial intelligence as a whole. It was made somewhat by taking inspiration from parts of biological neural networks.

Introduction to Machine learning - MP neuron
A lot of posts do a wonderful job in explaining the biological neural network so, I am not going to step into that territory. What we are going to in this post is build the intuition for upcoming posts of more sophisticated neural networks by learning how it all started.

1. Building blocks of a Neural Network
2. Threshold in Neural Network
3. Binary Data
4. Geometrical representation
5. How to calculate the threshold?
6. The MP Neuron class
7. Training and predicting a simple model
8. Conclusion
We are going to talk about things like, how MP neuron works. We will also, build the MP neuron class which we can use to train our models and predict simple things.

So, Let’s get started.

First, those who are interested in understanding the workings of biological neural network can check it on Wikipedia.

Let’s get back to the example that we used in the case of our Mathematics for Neural Network post.

Let’s say you are working with a YouTube video creator who makes unboxing videos of different phones and other electronic stuff. Your task is to predict which phone to review next so that your channel can get maximum views and maximum social media shares.

Now there are a lot of ways in which we can tackle this problem. We can go with a rule based system where we define which brands we want to review, whether we want to review phones with low cost, average cost or high cost.

The YouTube channel owner interviewed someone for his last YouTube video and told him that the MP neuron is the best way to make such predictions, and he should definitely ask his Data scientist to use MP neuron.

You tried to tell him about stuff like accuracy and other things, but he refuses to listen. So, you decided to show him by explaining him about the inner working of the neuron.

Let’s take an example of the video data that you have available and try building the intuition from there.

| Company  | Price | Number of Cameras | Screen length | Reviewed |
|----------|-------|-------------------|---------------|----------|
| Apple    | 1000  | 3                 | 5.5           |   1      |
| Samsung  | 800   | 1                 | 6             |   0      |
Reviewed is target value that we want to predict.

Building blocks of a Neural Network

In case of every Neural Network( even the ones we are going to talk about in the next few posts), we have a simple job, we will feed the data to the neuron and it will give us some output. For example, telling us whether we should review the phone or not.

This inner algorithm which helps a neuron to come to a conclusion is known as the Model.

Visually, if we consider only two input variables(for example Company and Price), it will look like,

MP neuron
Don’t get confused by the f(g(x)), it’s really simple.

g(x) stands for adding the input variables together on the other hand, f has this simple definition.

f(g(x)) is 1, if g(x) >= b
       and 0, if g(x) < b
This simply means that we are going to sum all the input variables, and if the sum of those variables is greater than or equal to some variable, b, we will return 1( or we will review the phone) else it will return 0( We will not review).

Threshold in Neural Network

Wait, did I forgot to define, b? Sorry, my bad.

b is term that we want to find so that we can find a clear separation between our classes( Reviewed and Not Reviewed in this case). It is also known as threshold/bias.

What are we going to do about the features having String values?

You might be asking yourself if we are going to sum stuff together, we won’t be able to use String features like, Company etc, right?

Well it is partially true.

Of course, we can’t use them directly, but we can convert the Company column to few different columns like, is_apple or is_samsung which will tell whether the phone was of Company Apple or not and whether the phone was of Company Samsung or not.

We can easily do this using Pandas,

pd.get_dummies(training_dataFrame(['Company']))
Therefore, we will end up having k different columns, if we have k different classes in the column.

If you don’t understand Pandas stuff, you might want to quickly, get a sneak peek using my post on Pandas( Self promotion never hurts).


Binary Data

Well, you might have guessed it till now and if you haven’t, we can only use features having binary data. If we don’t have binary features, we will have to convert it to Binary features.

For example, We can change Cost feature to Cost above > $1000 etc. If the cost is less than $1000, we will set the value to 0 and vice-versa.

Geometrical representation

Geometrically MP neuron is not very different from the Support Vector Machine model which also tries to divide the data using a line. The only thing missing in MP neurons is the Support Vectors.

The equation of the line is given by,

$x_1 + x_2 - b = 0$
This means that any point toward the one side of the line will produce +ve values and points on the other side will produce -ve values.

Extending it to the Reviewed/ Not Reviewed format, we will have all reviewed phones on one side of the line and not reviewed phones on the other.

How to calculate the threshold?

The main purpose of any model is to learn the correct values of the variables. b is the only variable in MP Neuron’s case. We will use a small trick so that we don’t have to check every possible value in the world to compute the correct threshold.

Every model is evaluated by comparing the true output with the predicted output to check how much accurate is our model. Mean Squared Error is one of those methods, which is also known as the loss function.

$\displaystyle \sum_{i}^{n}(y - \widehat{y})^2$
$where\ \widehat{y}\ is\ the\ predicted\ value.$
What we want to do is to minimize the value of the loss function.

We also are aware that the maximum sum that the model can have is equal to the number of features as we are only allowing binary features. Therefore, the value of threshold can vary from 0 to n.

We will calculate the corresponding loss at each data point and pick the threshold having the least loss.

The MP Neuron class

Now we will build the MP Neuron class that we can use to train our model. We will use the same format we were using before. A method called fit to fit the model and predict to predict the output.

import pandas as pd

import numpy as np
from sklearn.metrics import accuracy_score
import operator

class MPModel:
    def __init__(self, function='sum'):
        # We can pass some initial value of threshold.
        self.threshold = None
        if function == 'sum':
            self.function = self.sum_function
            
    def sum_function(self, x):
        return sum(x) >= self.threshold
    
    def and_function(self, x):
        return all(x)
    
    def or_function(self, x):
        return any(x)

    def fit(self, X_DataFrame, y_DataFrame):
        threshold_accuracy_dict = {}
        for threshold in range(len(X_DataFrame.columns) + 1):
            threshold_accuracy_dict[threshold] = None
        for threshold in threshold_accuracy_dict.keys():
            self.threshold = threshold
            predictions = self.predict(X_DataFrame)
            threshold_accuracy_dict[threshold] = accuracy_score(y_DataFrame, predictions)
        self.threshold = max(threshold_accuracy_dict.items(), key=operator.itemgetter(1))[0]
        print(self.threshold, 'threshold', threshold_accuracy_dict)

    def predict(self, X_DataFrame):
        results = np.array([])
        for i in range(len(X_DataFrame)):
            result = self.function(X_DataFrame.iloc[i])
            results = np.append(results, result)
        return results

Training and predicting a simple model

Consider the following data which we can use to train the MP Neuron model.

df_dict = {
    'Wind': [0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0],
    'Temp': [1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0],
    'Played': [0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0]
}
pd_df = pd.DataFrame(df_dict)


mpm = MPModel()
mpm.fit(pd_df[['Wind', 'Temp']], pd_df['Played'])

df_dict_test = {
    'Wind': [0, 1, 1, 0],
    'Temp': [1, 1, 0, 0],
    'Played': [1, 1, 1, 0]
}

pd_df_test = pd.DataFrame(df_dict_test)
predictions = mpm.predict(pd_df_test[['Wind', 'Temp']])
accuracy_score(predictions, pd_df_test['Played'])

> 1.0

Plotting learned values

import matplotlib.pyplot as plt

x = np.linspace(-0.5, 1.5, 100)
y = -x + 1
plt.plot(x, y, '-r', label='y=2x+1')
plt.plot()
plt.scatter(df_dict_test['Wind'], df_dict_test['Temp'], c=df_dict_test['Played'])
MP neuron plot
We can clearly see that the points on one direction of the line are marked with different color and points on the other side and on the line are marked with different colors.

I chose only 2 features to show this plot so that we can see the MP neuron in action.

We will not discuss anything related to what disadvantages does MP neuron have. We have moved the discussion to the next Neural Network in the list, which is a little better than this one.

Conclusion

These are the few takeaways of the MP Neuron.

  • First Model
  • Binary Input
  • Binary Output
  • Uses line to differentiate between datapoints.
  • Uses Mean Squared Error Loss function.
  • Sum all features to find the threshold.
In the next post, it will become clear to your partner that MP neuron is indeed not a good idea for the kind of problem you are trying to solve.

It is very important to start from thee basics if you want to learn something. This is the model from which everything started making much more sense. In the few upcoming post, it will become clear to you as well.

You May Also Enjoy

1. Normalizing or Standardizing distribution in Machine Learning
2. Basic Mathematics for Neural Networks | Vectors and Matrices with Matplotlib
3. SVM | Introduction to Support Vector Machines with Sklearn in Machine Learning
4. A simple mathematical guide to classification Trees using sklearn | Decision Trees
Please share your Feedback:

Did you enjoy reading or think it can be improved? Don’t forget to leave your thoughts in the comments section below! If you liked this article, please share it with your friends, and read a few more!

We don't share your details with others