Basic Elisp Regular Expressions

1. The Language of Patterns

At its heart, a regular expression is a string of characters that defines a search pattern. Some characters have special meanings (we call them metacharacters), while others are treated literally. To get a practical, Emacs-centric overview of these core components, we'll turn to a video by Protesilaos Stavrou, a prominent figure in the Emacs community.

The video is comprehensive, so we'll watch it in segments, focusing on one concept at a time. As you watch, don't worry about memorizing every detail immediately. The goal is to get a feel for what's possible.

Let's break down the key ideas from that video segment.

a. Anchors and Wildcards

These are some of the most common metacharacters.

b. Character Sets ([...])

Square brackets allow you to define a set of characters to match.

• Specific Characters: [aeiou] matches any single lowercase vowel.
• Ranges: [a-z] matches any lowercase letter. [0-9] matches any digit. You can combine them: [a-zA-Z0-9] matches any letter or digit.
• Negation: A caret ^ as the first character inside the brackets negates the set. [^0-9] matches any character that is not a digit.

c. Quantifiers (?, +, *)

These operators, called quantifiers or postfix operators, specify how many times the preceding character or group should appear.

d. Grouping and Alternation

These are crucial for building complex patterns.

• Grouping $...$: Parentheses group parts of a regex together, allowing you to apply a quantifier to the whole group. For example, $ha$+ matches "ha", "haha", "hahaha", and so on. Note the backslashes! This is an Emacs-specific syntax. Many other regex engines just use (...).
• Alternation \|: The escaped pipe character acts like an "OR". cat\|dog matches "cat" or "dog". When used in a group, like $cat\|dog$, it provides a choice within that part of the pattern. Again, note the backslash.

2. Your Interactive Regex Playground: re-builder

Reading about regex is one thing; using it is another. Emacs comes with a wonderful interactive tool called re-builder (Regular Expression Builder) that lets you type a regex and see it highlight matches in a buffer in real-time. This is the best way to experiment and solidify your understanding.

The following article from "Mastering Emacs" is a perfect introduction to this tool.

Hands-on Practice

Now it's time to put theory into practice.

1. Copy the text below into your *scratch* buffer.
2. With your cursor anywhere in the *scratch* buffer, run M-x re-builder.
3. A minibuffer will appear at the bottom. This is where you'll type your regular expressions.

Sample Text for Practice:

[2023-10-26] INFO: System startup complete.
[2023-10-26] WARN: Deprecated function 'old-style' called.
[2023-10-27] INFO: User 'anara' logged in.
[2023-10-27] ERROR: Failed to connect to database db-main-01.
[2023-10-28] ERROR: Failed to connect to database db-replica-02a.
This is a comment line that should not match patterns.
[2023-10-28] error: A minor, user-reported error.

Now, try to write a regex for each of the following challenges in the re-builder prompt.

• Challenge 1: Match only the lines that start with a timestamp (e.g., [2023-10-26]).
• Challenge 2: Match the words "INFO", "WARN", or "ERROR" but not "error".
• Challenge 3: Match all variants of "error" (i.e., "ERROR" and "error"). (Hint: re-builder is case-sensitive by default. Look at the article for how to toggle this, or use a character set.)
• Challenge 4: Match only the database names, which follow the pattern db- and end before the final period (.).

3. The "Backslash Plague": Regex in Elisp Strings

You've now seen that Emacs regex uses \( and \|. There is one final, crucial detail: writing a regex inside an Elisp string.

In an Elisp string, the backslash \ is also an escape character. To get a literal backslash that the regex engine can see, you must "escape the escape" by typing it twice: \\.

This means a regex that works in re-builder needs to have its backslashes doubled when you put it into a string in your code.

Let's make this concrete:

This is arguably the single most common point of confusion when writing Elisp that uses regular expressions. Forgetting to double the backslashes is a frequent source of bugs. A good habit is to always build and test your regex in re-builder first, and then use C-c C-w to copy it. Emacs will automatically convert it to the correct string format with doubled backslashes for you.

Conclusion

In this lesson, we've unpacked the syntax of basic regular expressions in Emacs Lisp. You've moved from the abstract idea of a "pattern" to writing a concrete language that describes it.

Key Takeaways:

• Regular expressions are a mini-language for describing text patterns.
• The core building blocks are anchors (^, $), wildcards (.), character sets ([...]), quantifiers (?, +, *), groups ($...$), and alternation (\|).
• Emacs regex syntax has quirks, most notably requiring backslashes for grouping and alternation (\( and \|).
• re-builder is an indispensable tool for interactively building and testing your patterns.
• When writing a regex inside an Elisp string, you must double all backslashes (e.g., \\().

Preview of the Next Lesson

You now know how to write a pattern. In our next lesson, "Use 'string-match' to find patterns and 'match-string' to extract matched groups," we will put these patterns to work. You'll learn how to use Elisp functions to search a string for a pattern and, more excitingly, how to extract the specific pieces of text that your pattern found. This will bring you one step closer to writing functions that can intelligently read and pull information from any text you give them.