Understanding K-Means Results

1. Understanding and Interpreting Centroids

In the previous lesson, we established that K-means works by finding cluster centers, or centroids. But what are they, exactly?

A centroid is the mean position of all the data points belonging to a particular cluster. Think of it as the cluster's "center of gravity." If your data has d features, each centroid is a d-dimensional vector where each component is the average value for that feature across all points in the cluster.

This is what makes centroids so powerful for interpretation: a centroid represents the "average" member of its cluster. By examining the coordinates of a centroid, you can build a representative profile for each group.

For a more formal definition of how centroids are calculated, this article provides a clear mathematical explanation.

Accessing and Visualizing Centroids in Scikit-Learn

After fitting a KMeans model, you can access the coordinates of the centroids using the cluster_centers_ attribute.

The following video segment demonstrates how to access these centroids and plot them alongside the data points.

The code to get and plot centroids looks like this:

# Assuming 'km' is your fitted KMeans model and 'plt' is matplotlib.pyplot

# Get the centroids
centroids = km.cluster_centers_

# Plot the centroids
plt.scatter(centroids[:, 0], centroids[:, 1], 
            marker='*', color='purple', s=250, label='Centroids')
plt.legend()
plt.show()

Interpreting Centroid Coordinates

A critical point to remember is that if you performed feature scaling (which you should!), the cluster_centers_ are reported in the scaled space. To make them interpretable in the context of your original data, you need to apply the inverse transformation.

Here’s how you would do that using a MinMaxScaler as in our previous lesson:

# Assuming 'scaler' is the fitted MinMaxScaler object
original_centroids = scaler.inverse_transform(km.cluster_centers_)

# For better readability, put them in a DataFrame
centroid_df = pd.DataFrame(original_centroids, columns=['Age', 'Income($)'])
print(centroid_df)

Running this would give you a table showing the "average" age and income for each cluster. For instance, you might find:

• Cluster 0: Low Age, Low Income
• Cluster 1: High Age, High Income
• Cluster 2: Middle Age, Middle Income

This is how you move from abstract cluster numbers to meaningful business insights, like identifying distinct customer segments.

2. A Deeper Look at Inertia (WCSS)

In the last lesson, we used the inertia_ attribute to create an elbow plot. Now, let's formally define it.

Inertia, also known as the Within-Cluster Sum of Squares (WCSS), is the sum of the squared distances from each data point to its assigned centroid.

The goal of the K-means algorithm is to find centroid positions that minimize this value. A lower inertia means the clusters are more compact and the points within each cluster are closer to their center.

The formula for inertia is:
$\text{Inertia}={\sum }_{i=1}^k{\sum }_{x\in C_i}\Vert x-{\mu }_i{\Vert }^2$
Where:

• $k$ is the number of clusters.
• $C_i$ is the set of points in cluster $i$.
• ${\mu }_i$ is the centroid of cluster $i$.
• $\Vert x-{\mu }_i{\Vert }^2$ is the squared Euclidean distance.

How to Interpret Inertia

Inertia is a measure of cluster cohesion (how tight the clusters are). However, it has key limitations:

• It's not a normalized metric. An inertia of 200 could be good for one dataset but poor for another. You cannot use it to compare clustering quality across different datasets.
• It always decreases as k increases. If you have as many clusters as data points (k=n), inertia will be 0. This is why we look for an "elbow" rather than just minimizing inertia.

So, the primary use of inertia is relative:

1. The Elbow Method: To compare results for different k on the same dataset to find an optimal k.
2. Internal Optimization: When you set n_init in scikit-learn's KMeans, the algorithm runs multiple times with different random initializations. It then returns the model from the run that achieved the lowest inertia for the given k.

3. Beyond Inertia: The Silhouette Score

Since inertia only measures cohesion and not separation, it can be misleading. A more comprehensive metric is the Silhouette Score, which measures how similar a point is to its own cluster compared to other clusters. It considers both:

• Cohesion: How close points are within a cluster.
• Separation: How far apart different clusters are.

The silhouette coefficient for a single data point i is calculated as:
$s(i)=\frac{b(i)-a(i)}{\max \{a(i),b(i)\}}$
Where:

• a(i) is the average distance from i to all other points in its own cluster (intra-cluster distance).
• b(i) is the minimum average distance from i to all points in any other cluster (inter-cluster distance).

The score ranges from -1 to +1:

• +1: The point is very dense in its cluster and far from others (ideal).
• 0: The point is on or very close to the boundary between two clusters.
• -1: The point is likely assigned to the wrong cluster.

The overall Silhouette Score for a clustering result is the average of the silhouette coefficients for all points. A higher average score indicates better-defined clusters.

To get a full grasp of this important metric, the following resource provides an excellent and detailed explanation.

Using Silhouette Score to Find the Optimal k

Unlike the Elbow Method, where we look for a bend, with the average silhouette score, we simply look for the k that gives the highest score.

Here is how you can calculate it in Python:

from sklearn.metrics import silhouette_score

silhouette_scores = []
k_range = range(2, 11) # Silhouette score is only defined for k > 1

for k in k_range:
    km = KMeans(n_clusters=k, n_init=10, random_state=42)
    km.fit(X_scaled)
    score = silhouette_score(X_scaled, km.labels_)
    silhouette_scores.append(score)

# Plot the results
plt.plot(k_range, silhouette_scores)
plt.xlabel('Number of clusters (k)')
plt.ylabel('Average Silhouette Score')
plt.title('Silhouette Score for Optimal k')
plt.show()

The k that corresponds to the peak of this plot is often a strong candidate for the optimal number of clusters.

Conclusion

You have now moved beyond just running the K-means algorithm to thoughtfully interpreting its results. By analyzing centroids and using robust evaluation metrics, you can turn raw clusters into actionable insights.

Key Takeaways:

• Centroids are the average of all points in a cluster. By inverse-transforming their coordinates, you can create a "profile" for each cluster.
• Inertia (WCSS) measures cluster compactness (cohesion). It is useful for the Elbow Method and for KMeans's internal n_init optimization, but it's not a standalone measure of quality.
• The Silhouette Score is a more powerful metric that evaluates both cluster cohesion and separation. The k that maximizes the average silhouette score is often a good choice.
• A thorough interpretation involves a combination of quantitative metrics (inertia, silhouette score) and qualitative analysis (examining centroids, visualizing results).

Preview of the Next Lesson:

A key characteristic of K-means is that we must specify the number of clusters, k, in advance. In our next lesson, we will explore hierarchical clustering, an alternative approach that doesn't require pre-specifying k. Instead, it builds a tree-like structure of nested clusters, which you can analyze using a visualization called a dendrogram. This will provide you with another powerful tool for discovering patterns in your data.