Counting with `collections.Counter`

The Manual Way: Counting with a Dictionary

A frequent programming task is to tally the occurrences of items in a sequence. For example, counting character frequencies in a string or tracking the number of times each word appears in a text.

Coming from a background where you might use a HashMap in Java, your first instinct in Python would be to use a dict. This involves iterating through the items and manually managing the counts.

Let's watch a video that demonstrates this evolution, starting with the most verbose, manual approach.

Let's break down the two approaches from the video.

Method 1: The "Look Before You Leap" (LBYL) approach

This is the most direct translation of the logic you might write in Java. You explicitly check if the key exists before you attempt to update its value.

# The "Bad" Way: Manual counting with if/else
words = ["apple", "banana", "apple", "orange", "banana", "apple"]
counts = {}

for word in words:
    if word in counts:
        counts[word] += 1
    else:
        counts[word] = 1

print(counts)



# Output: {'apple': 3, 'banana': 2, 'orange': 1}

This code is correct, but it's verbose. Python offers more expressive ways to handle this.

Method 2: A Better Way with dict.get()

As shown in the video, we can make this more concise by using the dict.get() method. get(key, default) returns the value for key if it's in the dictionary, otherwise it returns default. This eliminates the if/else block.

# A "Better" Way: Using dict.get()
words = ["apple", "banana", "apple", "orange", "banana", "apple"]
counts = {}

for word in words:
    counts[word] = counts.get(word, 0) + 1

print(counts)



# Output: {'apple': 3, 'banana': 2, 'orange': 1}

This is a common and useful Python idiom. However, for the specific task of counting, Python provides an even better, purpose-built tool.

The Pythonic Solution: collections.Counter

The "batteries-included" philosophy means that the standard library often has a specialized tool for common problems. For counting, that tool is collections.Counter. It's a dictionary subclass designed specifically for tallying hashable objects.

Let's see how Counter simplifies this task.

Counter handles the entire loop and logic for you. You simply pass it an iterable (like a list or string), and it does the rest.

# The "Good" / Pythonic Way: Using collections.Counter
from collections import Counter

words = ["apple", "banana", "apple", "orange", "banana", "apple"]
counts = Counter(words)

print(counts)



# Output: Counter({'apple': 3, 'banana': 2, 'orange': 1})

This single line of code is not only more readable and less error-prone, but as the video will show, it's also highly optimized and typically the fastest method.

Mastering the Counter API

Now that you understand why Counter is the right tool, let's explore its powerful API. It behaves like a dictionary but adds several convenient features.

This article from Real Python provides an excellent, in-depth guide. We will use it as our primary reference for this section.

Let's supplement that reading with a video that demonstrates these API features in action.

Practical Pattern: Counter with Generator Expressions

One of the most powerful ways to use Counter is by combining it with a generator expression. This allows you to perform a transformation on your data before it gets counted, all in one readable line.

Let's break down the second example from the image. Imagine you want to count words in a text but ignore punctuation and case.

The Manual Way:

text = "Hello, Python! What a wonderful world of Python."
words = []
for word in text.split():



    # Normalize the word before adding to list
    cleaned_word = word.strip(".,!").lower()
    words.append(cleaned_word)

counts = Counter(words)
print(counts)



# Output: Counter({'python': 2, 'hello': 1, 'what': 1, 'a': 1, 'wonderful': 1, 'world': 1, 'of': 1})

The Counter with Generator Expression Way:

text = "Hello, Python! What a wonderful world of Python."




# The generator expression `(word.strip(".,!").lower() for word in text.split())`
# processes each word on-the-fly as Counter consumes it.
counts = Counter(word.strip(".,!").lower() for word in text.split())

print(counts)



# Output: Counter({'python': 2, 'hello': 1, 'what': 1, 'a': 1, 'wonderful': 1, 'world': 1, 'of': 1})

This pattern is extremely common and is a hallmark of idiomatic Python code. It's concise, memory-efficient, and highly readable once you're familiar with the syntax.

Exercise: Refactor a Word Counter

Your task is to refactor a "bad" function that manually counts word frequencies into a "good" Pythonic function using collections.Counter.

The function should take a string of text, count the frequency of each word (case-insensitively, ignoring basic punctuation), and return the three most common words along with their counts.

from collections import Counter
import re

def get_top_3_words_manual(text: str) -> list[tuple[str, int]]:
    """
    A non-Pythonic function to find the top 3 most common words.
    - It's case-sensitive.
    - It doesn't handle punctuation well.
    - It manually sorts to find the top items.
    """
    counts = {}
    for word in text.split():



        # This is simple and flawed, we will improve it.
        word = word.lower() 
        if word in counts:
            counts[word] += 1
        else:
            counts[word] = 1
    



    # Manual sorting is complex and inefficient
    sorted_counts = sorted(counts.items(), key=lambda item: item[1], reverse=True)
    return sorted_counts[:3]


def get_top_3_words_pythonic(text: str) -> list[tuple[str, int]]:
    """
    A Pythonic function to find the top 3 most common words.
    - It should be case-insensitive.
    - It should strip punctuation from words.
    - It should use collections.Counter.
    """



    # Your implementation here.
    # Hint: Use a generator expression to clean the words before counting.
    # The regex re.findall(r'\b\w+\b', text.lower()) is a robust way to get words.
    words = re.findall(r'\b\w+\b', text.lower())
    word_counts = Counter(words)
    return word_counts.most_common(3)





# --- Verification ---
TEXT_SAMPLE = """
Python is a high-level, general-purpose programming language. Its design
philosophy emphasizes code readability with the use of significant indentation.
Python is dynamically typed and garbage-collected. It supports multiple
programming paradigms, including structured (particularly procedural),
object-oriented, and functional programming. Python is often described as a
'batteries included' language due to its comprehensive standard library.
"""




# Test the Pythonic function
top_3 = get_top_3_words_pythonic(TEXT_SAMPLE)
print(f"Top 3 words: {top_3}")




# Assertions to verify correctness
assert top_3[0][0] in ['python', 'is', 'a'] # Order can vary for same-count words
assert top_3[1][0] in ['python', 'is', 'a']
assert top_3[2][0] in ['python', 'is', 'a']
assert top_3[0][1] == 4 # 'python' appears 4 times
assert top_3[1][1] == 4 # 'is' appears 4 times
assert top_3[2][1] == 4 # 'a' appears 4 times




# Verify the next most common word is 'programming'
next_word, next_count = get_top_3_words_pythonic(TEXT_SAMPLE.replace("Python", ""))[2]
assert next_word == 'programming' and next_count == 2

print("\n✅ Exercise complete!")

Conclusion

In this lesson, you've seen how to move from a manual, verbose counting method to the specialized, efficient, and Pythonic collections.Counter. This embodies the principle of using the right tool for the job, a key aspect of the "batteries included" philosophy.

Key Takeaways:

• Replace Manual Loops: Counter(iterable) is the direct replacement for manual for loops that use a dict to tally items.
• Convenient API: Counter is a dict subclass, so it feels familiar, but it adds crucial features like returning 0 for missing keys and the .most_common() method.
• Readability and Performance: Using Counter makes your code more declarative and is almost always faster than manual implementations.
• Multiset Operations: Counter objects support arithmetic (+, -) and set-like (&, |) operations, making them powerful tools for comparing frequency distributions.

In our next lesson, we will continue exploring the collections module by looking at collections.defaultdict. You'll see how it elegantly solves the problem of grouping items into collections, another common task where manual if/else checks can be eliminated.