K-Means Clustering: Data Partitioning

1. Understanding the K-Means Algorithm

K-means is a centroid-based clustering algorithm. Its goal is to partition a dataset into a pre-determined number of clusters, which we call K. It achieves this by iteratively finding the best "center" or centroid for each cluster and assigning data points to the cluster with the nearest centroid.

The best way to understand this iterative process is to see it in action.

As the video demonstrated, the algorithm is conceptually straightforward. Let's formalize those steps:

1. Choose K: You must first decide how many clusters (K) you want to find in the data. We will explore methods for choosing an optimal K in our next lesson. For now, we will assume K is given.
2. Initialize Centroids: Randomly select K data points from your dataset to serve as the initial centroids.
3. Assign Points to Clusters: For each data point, calculate its distance to every centroid. Assign the data point to the cluster of the nearest centroid. The standard distance metric used is the Euclidean distance, which you'll recognize from geometry as the straight-line distance between two points. For two points $p=(p_1,p_2)$ and $q=(q_1,q_2)$ in 2D space, the distance is $\sqrt{(q_1-p_1{)}^2+(q_2-p_2{)}^2}$. This formula generalizes to any number of dimensions.
4. Update Centroids: Once all points are assigned to clusters, recalculate the position of each of the K centroids. The new position is the mean of all data points currently assigned to that cluster.
5. Repeat: Repeat steps 3 and 4 until the cluster assignments no longer change, or alternatively, the centroids stop moving significantly. At this point, the algorithm has converged.

Here is a static image that summarizes this iterative process visually.

2. The Importance of Feature Scaling

Before we apply k-means in code, we must address a crucial preprocessing step: feature scaling.

Because k-means relies on calculating distances, features with larger scales and variances will dominate the clustering process. For example, imagine you are clustering customer data based on two features: age (e.g., 20-70) and annual_income (e.g., 30,000-150,000). The sheer magnitude of the income values will dwarf the age values in the distance calculation. The algorithm would effectively ignore age and cluster the customers based almost entirely on their income.

To prevent this, we must scale our features so they are all on a comparable range. This ensures that each feature contributes fairly to the distance calculation.

This is a vital lesson in practical machine learning: for any distance-based algorithm (like K-means, PCA, and Support Vector Machines), feature scaling is not just recommended, it is essential for obtaining meaningful results.

3. Applying K-Means with Scikit-Learn

Now, let's walk through how to apply the k-means algorithm in Python using the popular scikit-learn library. We'll use the principles we've just discussed.

The process involves three main parts: preprocessing the data, building the k-means model, and examining the results.

Let's consolidate what you've seen in the video and the article into a clear workflow.

Step-by-Step Python Implementation

Here is a summary of the code structure you would use to perform k-means clustering.

1. Load and Prepare Data
First, you would load your data, for instance into a pandas DataFrame.

2. Preprocess the Data (Scaling)
As we've established, this step is critical. You'll create a scaler object and use it to transform your features. StandardScaler (which standardizes features to have a mean of 0 and a standard deviation of 1) is a common choice.

from sklearn.preprocessing import StandardScaler

# Assume 'features' is your numpy array or DataFrame with numerical data
scaler = StandardScaler()
scaled_features = scaler.fit_transform(features)

3. Instantiate and Fit the K-Means Model
Next, you create an instance of the KMeans class from scikit-learn. This is where you set the key parameters.

from sklearn.cluster import KMeans

# Instantiate the model
kmeans = KMeans(
    n_clusters=3,      # The number of clusters to find
    init="k-means++",  # Smart initialization, usually better than 'random'
    n_init=10,         # Run the algorithm 10 times with different random centroids
    max_iter=300,      # Max number of iterations for a single run
    random_state=42    # for reproducibility
)

# Fit the model to your scaled data
kmeans.fit(scaled_features)

A quick word on those parameters:

• n_clusters: The 'K' in K-means.
• init="k-means++": This is an advanced initialization technique that tends to speed up convergence and produce better results than pure random initialization. It's the default and generally recommended.
• n_init: Because the initial random placement of centroids can lead to different final clusters, scikit-learn runs the entire algorithm n_init times and returns the best outcome. This makes the result more robust.
• max_iter: A safeguard to prevent the algorithm from running forever if it doesn't converge.

4. Inspect the Results
After fitting, you can access the clustering results through the attributes of the kmeans object.

# The cluster assignment for each data point
cluster_labels = kmeans.labels_

# The coordinates of the final centroids
centroids = kmeans.cluster_centers_

# You can now add the labels back to your original DataFrame for analysis
# and plot the data, coloring points by their assigned cluster.

Visualizing the clusters and centroids, as shown in the codebasics video, is the best way to interpret your model's output.

Conclusion

Congratulations! You've just learned how to apply one of the most foundational algorithms in unsupervised machine learning. You now have the skills to take a raw dataset, preprocess it correctly, and partition it into meaningful groups.

Key Takeaways:

• K-Means is an iterative algorithm that partitions data into K clusters by minimizing the distance between data points and their cluster's centroid.
• The algorithm follows a simple loop: initialize centroids, assign points, and update centroids until convergence.
• Feature scaling is mandatory for distance-based algorithms like K-means to ensure all features contribute fairly to the result.
• scikit-learn provides a robust KMeans implementation with important parameters like n_clusters, n_init, and init that help you build a reliable model.

Preview of the Next Lesson:

Throughout this lesson, we have manually specified the number of clusters, K. But in a real-world problem, how do you know if you should be looking for 2, 3, or 10 clusters? This is a critical question in clustering. In our next lesson, we will address it by learning how to select the number of clusters for k-means using the elbow method or silhouette scores.