Image Classification using Knn

 

What is knn:

The k-nearest neighbours algorithm (k-NN) in statistics was created by Evelyn Fix and Joseph Hodges in 1951 and later improved by Thomas Cover. It is a non-parametric supervised learning technique. Regression and classification are two uses for it. The input in both situations consists of a data set's k closest training samples. Whether k-NN is applied for classification or regression determines the results:

The result of k-NN classification is a class membership. The class that an object is assigned to based on the majority vote of its k closest neighbours is determined by the item's neighbours (k is a positive integer, typically small). The object is simply put into the class of its one nearest neighbour if k = 1.

The output of k-NN regression is the object's property value. The average of the values of the k closest neighbours makes up this number.

Statistical setting

Suppose we have pairs ${\displaystyle (X_{1},Y_{1}),(X_{2},Y_{2}),\dots ,(X_{n},Y_{n})}$ taking values in ${\displaystyle \mathbb {R} ^{d}\times \{1,2\}}$, where Y is the class label of X, so that ${\displaystyle X|Y=r\sim P_{r}}$ for ${\displaystyle r=1,2}$ (and probability distributions ${\displaystyle P_{r}}$). Given some norm ${\displaystyle \|\cdot \|}$ on ${\displaystyle \mathbb {R} ^{d}}$ and a point ${\displaystyle x\in \mathbb {R} ^{d}}$, let ${\displaystyle (X_{(1)},Y_{(1)}),\dots ,(X_{(n)},Y_{(n)})}$ be a reordering of the training data such that ${\displaystyle \|X_{(1)}-x\|\leq \dots \leq \|X_{(n)}-x\|}$

Algorithm

The training examples are vectors with class labels in a multidimensional feature space. Only the feature vectors and class labels of the training samples are stored during the algorithm's training phase.

The label that appears most frequently among the k training samples that are the closest to the unlabeled vector (a query or test point) during the classification phase, where k is a user-defined constant, is assigned to the query point.

Euclidean distance is a typical distance metric for continuous variables. Another metric, such as the overlap metric, can be used for discrete variables, such as text categorization (or Hamming distance). For instance, k-NN has been used as a metric in the context of gene expression microarray data along with correlation coefficients like Pearson and Spearman. If the distance metric is learned with specific algorithms like Large Margin Nearest Neighbor or Neighbourhood Components Analysis, the classification accuracy of k-NN can frequently be greatly increased.

When the distribution of the classes is skewed, the fundamental "majority voting" categorization has a disadvantage. In other words, because they tend to be common among the k nearest neighbours due to their high quantity, examples of a more frequent class tend to dominate the forecast of the new example. To solve this issue, one solution is to weight the classification by accounting for the distance between the test location and each of its k closest neighbours. Each of the k closest points' class (or value, in regression issues) is multiplied by a weight that is proportional to the inverse of the distance separating it from the test point. Abstraction in data representation is a different strategy for dealing with skew. In a self-organizing map, for instanceRegardless of their density in the initial training data, (SOM), each node is a representation (a centre) of a cluster of comparable points. The SOM can then be used with K-NN.

How to predict image by Knn:

The k-NN algorithm categorises unknown data points by identifying the most prevalent class among the k-closest examples, to put it simply. The category with the most votes wins, and each data point in the k closest examples casts a vote.

WATCH MY VIDEO