SVM | Introduction to Support Vector Machines with Sklearn in Machine Learning
SVM or support vector machines are supervised learning models that analyze data and recognize patterns on its own. They are used for both classification and regression analysis. An SVM model is the representation of the dataset as points in space so that the example of the separate categories is divided by a clear gap which is as wide as possible.
2. Support vector classifier
3. Support Vector Machines
4. Support Vector Machine for more than two classes
5. Support Vector Machine using Sklearn
- Maximal Margin Classifier
- Support Vector Classifier
- Support Vector Machine
Maximal Margin Classifier
Maximal Margin Classifier is a model that is used to classify the observations into two parts using a hyperplane.
What is a Hyperplane?Simply put, a hyperplane is a subspace in
p -1dimensions. For example, in two-dimensional space, the hyperplane will be of 1 dimension, or it will be a line. Similarly, in the case of 3 dimensions, it will be a two-dimensional plane. In two dimensions the equation of the hyperplane are given by,
n X pmatrix
ntraining observations in
p-dimensional space such that all these vectors falls in one of the two classes divided by the hyperplane. If we represent the classes(labels) for all the n values,
marginfrom both the sides is chosen as the hyperplane.
pis large. We have already discussed that the points on dashed line are called support vectors and it has been found that the position of hyperplane only depends on support vector and is not dependent on the other observations in the dataset. This is how we define a Maximal margin classifier. There are a few issues with Maximal Margin Classifier.
- It doesn’t work on observations where no clear hyperplane is present between different classes.
- A small addition of observation near the hyperplane can lead to a lot of change in the hyperplane making it a lot volatile.
Support vector classifierIn case of Support vector classifiers, we allow a few observations to be on the wrong side of hyperplane making the model a little more robust to individual observation and helps us to better classify other and most of the observations.
Support vector classifier is also known as a soft margin classifier.The observations on the wrong side of the hyperplane are obviously misclassified by the model. But this helps to improve the overall accuracy of the model.
epsilon( also called a slack variable), gives the point at which it is located, relative to the hyperplane and margin. If
ith slack variable is on the right side of the hyperplane then the value of that variable is 0. Also, if
iis on the wrong side of the margin. But if,
C, we can deduce that
Cis the number which determines the count and severity of the violations to the margins and the hyperplane. The value of
Cis considered as the tuning parameter which is generally chosen by cross-validation.
Calso controls the bias-variance trade-off for the model. If the value of
Cis small, we allow a lesser number of observations to be on the wrong side which will fit perfectly to a data set having data with high bias and low variance and vice-versa. Again similar to Maximal Margin Classifier, it was found that all the observations don’t get to decide the position of a hyperplane of the Margin. It is only dependent on the observations on or inside the margins. If we expand these points a little we can get to the Support Vector Machines. Let’s discuss them in some detail.
Support Vector MachinesIn the Support vector Machine, we introduce another factor called the kernel, which is the result of enlarging of support vector classifiers in a specific way. According to our discussions in support vector classifier, its equation can be re-written as,
The implementation of the inner product is hidden on purpose and we should be good without knowing the details of it.We can directly replace all the instances of the inner product with a general term called the kernel.
pplanes equation of kernel becomes,
d. This type of model leads to much flexible decision boundary.
Support Vector Machine for more than two classesDuring the discussion for support vector machines, we haven’t really talked about the case when the number of possible classifications can be more than 2. We can solve these problems by extending the simple SVM in two ways.
- One versus One Classification
- One versus All Classification
Kclass to the remaining
K - 1classes. Finally, we will assign any upcoming test vector to the class which produces the highest values of the constant, or we want to maximize.
Support Vector Machine using SklearnNow we will try to train a Support Vector Machine model using sklearn. We are going to work on the already available cancer data which we have used in other posts as well. So, the idea behind the data is that, we have been given some cancer patients info, for example what is the perimeter, radius and whole lot of other stuff about the cancer. Our goal is to find whether the cancer is
WDBC-Benign. Let’s import everything first.
import pandas as pd import numpy as np
from sklearn.datasets import load_breast_cancer cancer = load_breast_cancer() df = pd.DataFrame(cancer['data'], columns=cancer['feature_names']) df.head()
from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(df, cancer['target'], test_size=0.40)
from sklearn.svm import SVC model = SVC() model.fit(X_train, y_train)
from sklearn.metrics import classification_report from sklearn.metrics import confusion_matrix predictions = model.predict(X_test) print(confusion_matrix(y_test, predictions)) print(classification_report(y_test, predictions))
Did you enjoy reading or think it can be improved? Don’t forget to leave your thoughts in the comments section below! If you liked this article, please share it with your friends, and read a few more!