Support Vector Machines- I

 Support Vector Machine (SVM):

It is a supervised Machine Learning Algorithm. It is used to Classification and Regression model. 

What is SVM

SVM is a Machine Learning algorithm. It creates the best line or decision boundary (Hyper plane) that can separate out n-dimensional space into classes. The decision boundary is used to classify new data.

  1.   SVM is a type of supervised Learning Algorithm.
  2.    It requires labeled data (Input and output both) for training.
  3.    It is used to solve classification and regression problems.        

When should we use a Support Vector Machine?

Support vector machine is used two classify two-dimensional and multidimensional data and also regression model.

Decision Boundary: Decision boundary (Line or Plane) plays important role in the classification.


                        Figure1: Hyperplane and line Classify the data                                                              


To classify blue and red dots Support vector creates a decision boundary between these two data sets.

If it is multidimensional data, plane is used and if it is two-dimensional data line is used. This line or a plane is called Hyperplane as shown in figure1

Following are the important concepts in SVM

Define following concept.

1.   Linearly and non linear SVM

2.   Hyper Plane.

3.   Distance Margin

4.   Support Vectors

1.   Types of SVM

SVM can be of two types:

  • Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used called as Linear SVM classifier. Figure 2 is an example of Linear SVM.

  • Figure 2: Linear Classification

     

    Non-linear SVM: Non-Linear SVM is used to classify non-linearly separated data, which means if a datasets not be classified  by using a straight line, then such data is termed as non-linear data

  • Figure 3: Non Linear SVM

    Figure 3 is an example of Non Linear SVM. In figure 3.It is not possible to draw a line to separate blue and yellow spot. Plane is used to classify data. To ceat plane, need to convert into a multidimensional plot using kernal. Plane is created using kerne to classify data

  • Figure 4: Plane to separate non linear data

    Difference between Linear SVM and Non Linear SVM

     

    Linear SVM

    Non-Linear SVM

     Linear line will separate the data and classify

    Linear line will not  separate the data

    Data is classified with the help of hyperplane.

     Kernels are used to make non-separable data into separable data.

    Data can be easily classified by drawing a straight line.

    Need to map data from low dimensional space to  high dimensional space for classification.

     

    Summary of Linear SVM and Non-Linear SVM.

    • In Linear SVM Data set can be classified into two classes by using a single straight line, 
    • In Non Linear SVM Data set cannot  be classified into two classes by using a single straight line
    • In Non Linear classifier plane is used to classify the data.
    • Hyperplane(Line/Plane) is used to classify data in SVM
    • Kernels are used to classify non linear Data.
    • Kernels are used to map data from low dimensional space to high dimensional space for classification.
    • Kernels  are used to make non-separable data into separable data in non linear classifier

    1.   What is Hyperplane? 

    A most important step to build a  SVM Model  is to create Hyperplane which has a maximum margin

    Hyperplane: 

    Hyperplane is a decision plane or space that divides set of objects having different classes between the data points. There can be multiple lines/decision boundaries to seperate the classes in n-dimensional space, but best decision boundary need to find out that helps to classify the data points. This best boundary is known as the hyperplane of SVM.In figure 4 large margin is the best boundary.

  • Figure 5: Hyperplane

    1.   Margin Maximum margin hyperplane is a decision plane or space that divides set of objects having different classes between the data points. We always create a hyperplane that has a maximum margin, which means the maximum distance. It can be easily seen that line from figure 4 with a large margin is the one that best separates the two classes. 

    2.   Support Vector- The data points on hyperplane boundary is called Support Vector. Figure 6 shows Hyper plane, Margin, Support Vector.

     

  • Figure 6: Hyperplane. Margin, Support Vector

     

    To build SVM Model the most important part is to choose the hyperplane which is as far as possible from the data points of each category.

    How Doeds SVM Works?

    Step1: Collect data set of green and blue dot. It is supervised learning with labels green for green dot and blue for  blue dot.

    Step 2: Apply SVM .It creates hyperplane. It is two diamensional data Hyperplane is line which separate green and blue dots.

    Step 3:Distance between positive hyperplane and Negative hyperplane should be maximum hen it is maximum margin hyperplane.Check Hyperplane is a maximum margin hyperplane. 

    Step 4: Find Support Vectors.

    Step 5: Support vectors are used to classify the new points.



What are the advantages of supporting Vector Machine over other classification problem?

SVMs are extremely popular because of their ability to handle multiple continuous and categorical variables.

Disadvantages of Neural Network over SVM

1.   Having many local minima

2.   To find number of neuron and number of layers in the architecture.

3.   Trial and error method  to decide structure and hyperparameters  to reach optimality of that NN 

There are many linear classifiers (hyper planes) that separate the data. However the goals of SVM are separating the data with hyper plane and extend this to non-linear boundaries using kernel trick. 

Advantages of SVM are

  It is  maximum margin classifier or hyper plane.  It gives  maximum separation between two class

·         Text and Hypertext category

·         Classification of Images

·         Classification of satellite data.

·         Recognition of hand written character

·         Biological and other sciences to classify proteins with up to 90% of the compounds classified correctly. 

Comments

Post a Comment

Popular posts from this blog

Linear Regression

Image
Linear Regression and Gradient Decent method to learn model of Linear Regression.   Linear Regression Linear regression: Machine Learning Algorithm  Variables in Linear regression Independent variable Ø  If X is input numerical variable then X is called the independent variable or predictor.It is input of the model. All Features /Co-variant are independent variable   Dependent Variable Ø  If Y is output numerical variable. Y is also called the dependent variable or response variable. It Output of a model Machine learning models are built to derive the relationship between the dependent variable and independent variable.  It predicts a continuous dependent variable based on values of the independent variable in case of  Linear Regression Ø  Linear regression is Supervised Learning. It predicts Relationship between dependent and an independent variable which is linear Ø  e.g Income *  Expendit...
Read more

k-Nearest Neighbors Algorithm

Image
    •       K-Nearest Neighbor is one of the simplest Machine Learning algorithms based on Supervised Learning techniques. •      K-NN algorithm assumes the similarity between the new case/data and available cases and puts the new case into the category that is most similar to the general categories. •       K-NN algorithm stores all the available data and classifies a new data point based on the similarity. This means when new data appears it can be easily classified into a well-suited category by using the K-NN algorithm. •       K-NN algorithm can be used for Regression as well as for Classification but mostly it is used for Classification problems •       K-NN is a  non-parametric algorithm , which means it does not make any assumption on underlying data. •       It is also called a  lazy learner algorithm  because it ...
Read more

Logistic Regression

Image
  Top of Form Bottom of Form   Logistic Regression ·          Logistic regression is widely used for binary classification problems. It gives binary output. A binary dependent variable can have only two values, like 0 or 1, win or lose, pass or fail, healthy or sick, etc. ·          Logistic Regression: Output is in Binary ·          It can also be extended to multi-class classification problems whose output can be classified into more than one category. ·          Logistic regression can be used to predict a dependent variable(output) on the basis of continuous and/or categorical independents(input) ·          The dependent variable(output) is categorical: y ϵ {0, 1} ·          It is scalable...
Read more