Python實現KNN演算法

    KNN演算法的實際用處很多，主要用於分類階段，是一個基礎的分類演算法。KNN主要基於距離的計算，一般可以在原始的歐氏空間中計算樣本之間的距離。改進版本有：先特徵提取到一個更加鑑別的空間中，然後計算距離；或者先使用metric learning度量學習的技術來獲得一個鑑別的度量空間，然後計算樣本間的馬氏距離。
    不管怎麼說，KNN在很多演算法的分類階段都可以用到，我們這裡用python實現KNN。

1. sklearn自帶的KNN

fromsklearn.neighborsimport NearestNeighbors

就可以呼叫最近鄰演算法了。

'''
python實現KNN演算法
'''

#只是返回近鄰點，不分類
from sklearn.neighbors import NearestNeighbors  #載入最近鄰演算法
samples = [[0, 0, 0], [0, 0.5, 0], [1, 1, 0.5]]; 
neigh = NearestNeighbors(n_neighbors=2)  #set the number of neighbors 
neigh.fit(samples) 
print neigh.kneighbors([1, 1, 1]) #return the same number of neighbors 
#return two arrays, the first is the calculated distance; the second is the indexs of neighbors, strarting from 0 



#實現分類
from sklearn.neighbors import KNeighborsClassifier 
knnclf = KNeighborsClassifier(n_neighbors=1)  #we set the k=1, while default with k=5 
samples = [[0, 0, 0], [0, 0.5, 0], [1, 1, 0.5]]  #training samples features
labels = [0, 0, 1]  #the labels 
knnclf.fit(samples, labels) 
print knnclf.predict([1, 1, 1])  #return the classification label, that is, [1]

2. 原始碼實現

    我先自己用實現了一遍，然後再看它的原始碼，對比發現對python的使用還有待提高！

    自己實現的KNN程式碼：

#編碼實現KNN

from numpy import * 
import operator 

def creatDataset(): 
    samples = [[0, 0, 0, 0.5], [0, 0.5, 0, 1], [1, 1, 0.5, 0]]  #training samples features
    samples = mat(samples) 
    labels = [0, 0, 1]  #the labels 
    return samples, labels 

def kNNClassifier(traSamples, lables, k, tstSample): 
    samNum,feaDim = shape(traSamples);  # each line is one sample 
    minDist = 10
    classifiedLabel = labels[0]
    for i in range(samNum): 
        tmpDist = (traSamples[i] - tstSample) * (traSamples[i] - tstSample).T  # notice that tmpDist is a matrix here
        print tmpDist 
        if(tmpDist[0][0] < minDist):  # since tmpDist is a matrix 
            minDist = tmpDist 
            classifiedLabel = labels[i] 
    return classifiedLabel 

tstSample = mat([[1, 1, 1, 0]] )
samples, labels = creatDataset()
print kNNClassifier(samples, labels, 1, tstSample) 

def classify0(inX, dataSet, labels, k): 
    dataSetSize = dataSet.shape[0] # the number of samples 
    
    # tile function is the same as "replicate" function of MATLAB
    # 這個技巧就避免了迴圈語句
    diffMat = tile(inX, (dataSetSize, 1)) - dataSet # replicate inX into dataSetSize * 1
    sqDiffMat = diffMat**2  # 對應元素平方
    sqDistances = sqDiffMat.sum(axis = 1)  # 按行求和
    distances = sqDistances**0.5  # 開方求距離
    
    sortedDistIndicies = distances.argsort()  # argsort函式返回的是陣列值從小到大的索引值
    classCount = {} 
    # 投票
    for i in range(k): 
        voteIlabel = labels[sortedDistIndicies[i]] #排名第i近的樣本的label
        classCount[voteIlabel] = classCount.get(voteIlabel, 0) + 1  #get字典的元素，如果不存在key，則為0
    # operator.itemgetter(1)按照value排序；也可以用 key = lambda asd:asd[1]
    # 排序完，原classCount不變
    sortedClassCount = sorted(classCount.iteritems(),  # 鍵值對
                              key = operator.itemgetter(1), reverse = True)  #逆序排列 
    
    return sortedClassCount[0][0]  #輸出第一個，也就是最近鄰