Clustering

• Goal : Group a collection of objects into clusters, such that those within each cluster are more closely related to one another than objects assigned to different clusters.

• There is no information about the class in data. There is no reponse variable (label information)

• Popular clustering algorithms

• k-means

Uses squared Euclidean distance as a dissimilarity measure.

All Varibales must be quantitative (numeric)

Objective function : Try to minimize the total within scatter (sum of within sum of squares)

The results depend on the initial values : Try to run the algorithm several times with random initial values to achieve global minima.

R function : kmeans(x,k)    x:data matrix, k:number of clusters

 

• PAM

Any distance can be used. Need a distance matrix as an input

Therefore, Any types of variables in X can be used.

Similar objective function as k-means: Minimize total within scatter

Computationally more intensive then k-means

R function : pam(dist(x),k) or pam(x,k)    read help file. needs package "cluster"

• Hierarchical clustering

Bottom-up clustering methods

Starts with n clusters and ends with 1 cluster.

Finds 2 closest objects, merge them find next closest, merge them until all objects are merged.

Needs a definition of distance between groups: most popular : Single Linkage, Complete Linkage, Group Average

R function : hclust(dist(x))