What Is Clustering in Machine Learning?

Clustering groups data points that resemble each other, without anyone telling the algorithm what those groups should be.

People generally find clustering to be intuitive because it’s closely aligned to how we think as humans. We’re trying to simplify the world by identifying patterns and creating frameworks to group what we’re observing. Let’s start our clustering example by running the code below.

#import required libraries

import numpy as np

from sklearn.model_selection import train_test_split

from sklearn.cluster import KMeans

from sklearn.datasets import make_blobs

import matplotlib.pyplot as plt

from sklearn.metrics import silhouette_score

#create sample data

n_samples = 300

random_state = 42

X, y = make_blobs(

    n_samples=n_samples,

   random_state=random_state

   )

#execute train test split

X_train, X_test = train_test_split(

   X,

   test_size=0.3,

   random_state=random_state

   )

#set number of clusters to 3

n_clusters = 3

#create kmeans clustering model

kmeans = KMeans(

   n_clusters=n_clusters,

   random_state=random_state,

   n_init='auto'

   )

#fit the model to the data

kmeans.fit(X)

#create the predictions on the test data

test_labels = kmeans.predict(X_test)

#create the predctions on the training data

train_labels = kmeans.predict(X_train)

#calculate and print the silhouette score for the training data

silhouette_avg = silhouette_score(X_train, train_labels)

print(silhouette_avg)

#calculate the silhouette score for the test data

silhouette_avg = silhouette_score(X_test, test_labels)

print(silhouette_avg)

The code is importing new packages for clustering. Kmeans() is the most common approach to clustering. While there are others, our focus will be on the kmeans() algorithm. Let’s discuss this code in more detail:

Creating Dataset

We’re generating our own data for this example, and sklearn provides us with the make_blobs() function to do so.

Slightly Different Splitting of Data

Clustering is unsupervised learning, meaning it doesn’t contain a y portion to split. This is why, we’re only inputting and outputting the x components in our train_test_split() function.

Clustering Algorithm

The clustering algorithm has two new hyperparameters we need to learn. The first is n_clusters, which says how many groups need to be created from the data. The next is n_init, which identifies how many different starting points the algorithm will try.

Silhouette Metric

We have a new metric from the silhouette_score() function. At a high level, this measures the quality of the clusters by comparing the distance of the data points within a cluster to the data points of the next closest clusters. The score can be anywhere from -1 to 1, with 1 representing very good clusters and -1 representing clusters that may be incorrect. A 0 usually represents clusters that overlap. The output from this code is 0.84 on the training data and 0.86 on the test data, which is pretty good!

Clustering tends to be visual, so we can also visualize our results as shown in the code below with the output shown in the figure. Using the plt.scatter function, we take our centers dataset and provide it with the x and y coordinates (centers[:,0] and centers [:, 1]) to plot the data.

#create predictions for full dataset

all_labels = kmeans.predict(X)

#plot the data in a scatterplot

plt.scatter(X[:, 0], X[:, 1], c=all_labels, s=50, cmap='viridis')

centers = kmeans.cluster_centers_

plt.scatter(centers[:, 0], centers[:, 1], c='red', s=200, alpha=0.75)

plt.title('K-means Clustering')

plt.xlabel('Feature 1')

plt.ylabel('Feature 2')

plt.show()

Conclusion