What are kernels in machine learning and SVM and why do we need them?

Found this on Reddit: Please explain Support Vector Machines (SVM) like I am a 5 year old. • /r/MachineLearning
Simply the best explanation of SVM i ever found.
----------------------------------------------------------------------------------------------
We have 2 colors of balls on the table that we want to separate.
We get a stick and put it on the table, this works pretty well right?
Some villain comes and places more balls on the table, it kind of works but one of the balls is on the wrong side and there is probably a better place to put the stick now.
SVMs try to put the stick in the best possible place by having as big a gap on either side of the stick as possible.
Now when the villain returns the stick is still in a pretty good spot.
There is another trick in the SVM toolbox that is even more important. Say the villain has seen how good you are with a stick so he gives you a new challenge.
There’s no stick in the world that will let you split those balls well, so what do you do? You flip the table of course! Throwing the balls into the air. Then, with your pro ninja skills, you grab a sheet of paper and slip it between the balls.
Now, looking at the balls from where the villain is standing, they balls will look split by some curvy line.
Boring adults the call balls data, the stick a classifier, the biggest gap trick optimization, call flipping the table kernelling and the piece of paper a hyperplane.
-----------------------------------------------------------------------------------------------
Now see this:
--------------------------------------------------------------------------------------------
One other point that I like to mention about the SVM (unrelated to this question) is how it is defined by the boundary case examples. (taken from CS109 course:SVM Explanation)
Assume you want to separate Apples from oranges using SVM. The Red square are Apples and the blue circles are oranges.
Now see the support vectors(The filled Points) which define the margin here.
Intuitively the filled Blue circle is an Orange that looks very much like an Apple. While the filled Red squares are Apples that very much look like oranges.
Think for this another time. If you would want your kid to learn to differentiate between an apple and orange you would show him a perfect apple and a perfect orange. But not SVMs they want to only see an apple that looks like an orange and vice versa. This approach is very different from how most Machine learning algorithms operate, and maybe thats why it works so well in some cases.
If you want to learn more about Data Science and Machine Learning, I would like to call out this excellent course by Andrew Ng. This was the one that got me started. Do check it out.
Also, I will continue writing beginner friendly data science posts. Follow me up at Medium or Subscribe to my blog to be informed about them.

Comments

Post a Comment

Popular posts from this blog

Maxpooling vs minpooling vs average pooling

Image
Pooling is performed in neural networks to reduce variance and computation complexity. Many times, beginners blindly use a pooling method without knowing the reason for using it. Here is a comparison of three basic pooling methods that are widely used. The three types of pooling operations are: Max pooling: The maximum pixel value of the batch is selected. Min pooling: The minimum pixel value of the batch is selected. Average pooling: The average value of all the pixels in the batch is selected. The batch here means a group of pixels of size equal to the filter size which is decided based on the size of the image. In the following example, a filter of 9x9 is chosen. The output of the pooling method varies with the varying value of the filter size. The operations are illustrated through the following figures. Average, Max and Min pooling of size 9x9 applied on an image We cannot say that a particular pooling method is better over others generally. The choice of pooling operation is made...
Read more

Generative AI - Prompting with purpose: The RACE framework for data analysis

Image
  Prompting with purpose: The RACE framework for data analysis We've already established that the quality of your prompts directly influences the quality of your AI-generated outputs. A well-crafted prompt acts as a guiding light, leading the AI toward insightful and actionable results. But how can we consistently achieve this level of clarity and precision in our prompts? Let’s look into the RACE framework—an invaluable tool to elevate your prompts, ensuring they are clear, insightful, and evidence-based. This framework offers a structured approach to prompt engineering, helping you break down complex tasks into manageable steps and communicate your intentions effectively to the AI. This reading will assist you with developing a powerful methodology with generative AI for your data analysis tasks, leading to more accurate, relevant, and impactful results. What is the RACE framework? RACE is an acronym that stands for Research, Answer, Cite, and Expand. It serves as a valuable fram...
Read more

Best Practices for Storing and Loading JSON Objects from a Large SQL Server Table Using .NET Core

 Best Practices for Storing and Loading JSON Objects from a Large SQL Server Table Using .NET Core Introduction JSON (JavaScript Object Notation) is widely used for storing structured data in SQL Server. However, when dealing with large tables, inefficient handling of JSON data can lead to performance bottlenecks, high storage costs, and slow queries. In this article, we will cover best practices for saving and loading JSON objects from large SQL Server tables using .NET Core. We will focus on storage strategies, indexing, efficient querying, and optimized data retrieval using Entity Framework Core (EF Core) and Dapper. --- 1. Choosing the Right Storage Strategy SQL Server supports JSON natively, but choosing the correct storage method depends on your use case. Option 1: Store JSON as NVARCHAR(MAX) (Best for Flexibility & Simplicity) ✔ Ideal for semi-structured or frequently changing data. ✔ Suitable when you need to store the entire JSON object but rarely query individual fiel...
Read more