List Concatenation: Append vs. Nconc

Introduction

Welcome to the next lesson on list manipulation in Elisp. In our previous session, we explored how to modify a variable holding a list using push and pop. We saw that these macros are convenient for managing stacks, but importantly, they work by reassigning the variable to a new list, leaving the original list structure intact for any other variables that might be pointing to it. We also previously saw that append concatenates lists non-destructively, creating a new list from its arguments.

Today, we will dive into a concept that you specifically wanted to master: the distinction between non-destructive and truly destructive operations. This lesson directly addresses the learning outcome: Distinguish between non-destructive ('append') and destructive ('nconc') list concatenation.

We will compare the familiar append with its powerful but perilous counterpart, nconc. Understanding the difference is not just an academic exercise; it's fundamental to writing efficient, predictable, and bug-free Elisp code. This knowledge gives you finer control over memory and performance, a hallmark of the Lisp family of languages.

1. A Quick Refresher: `append`

Let's start with what you already know. The append function takes any number of lists and creates a new list containing all their elements in order.

Crucially, append is non-destructive. It works by making copies of all but its last argument and linking them together. The original lists you pass to it are left completely unchanged.

Let's verify this in your *scratch* buffer.

Create two lists. We can imagine these are lists of themes for a speculative fiction story.

(setq themes-tech '("AI consciousness" "biopunk"))
(setq themes-social '("dystopian governance" "post-scarcity economics"))

Evaluate both setq forms.

Now, use append to create a combined list.
```
(setq all-themes (append themes-tech themes-social))
```
Evaluate this. The variable all-themes now holds the list ("AI consciousness" "biopunk" "dystopian governance" "post-scarcity economics").
Finally, let's inspect the original list, themes-tech.
```
themes-tech
```
Evaluating this shows ("AI consciousness" "biopunk"). It is completely unaffected. The append function worked on a copy, preserving the original. This is the safe and predictable way to concatenate.

2. Destructive Concatenation with `nconc`

Now, let's meet nconc (the "n" stands for "non-consing," meaning it doesn't create new cons cells). It is the destructive counterpart to append.

Instead of copying lists, nconc modifies the lists in-place. Specifically, it takes the last cons cell of the first list and changes its cdr (the "rest" part of the pair) to point directly to the beginning of the second list.

The result is a single, altered list. This is much faster and uses less memory than append, as no copying occurs. However, this efficiency comes at the cost of safety.

The following image provides an excellent visualization of what happens.

NCONC (Destructive Append) Visual Explanation — This diagram shows two lists, `x` and `y`. The `nconc` function doesn't copy `x`. Instead, it finds the last cons cell in `x` (the one containing 'lade') and physically alters its `cdr` pointer (originally `nil`) to point to the first cons cell of `y` (the one containing '1'). As a result, the variable `x` itself now refers to the modified, longer list.

Let's see this "spooky action" for ourselves.

GNU Emacs Lisp Reference Manual - Lists

The GNU Emacs Lisp Reference Manual provides a clear definition and example of nconc. This reading will show you its fundamental behavior.

Please read the first part of the section 'Functions that Rearrange Lists', starting from the description of nconc. Focus on the first code example with the variable x. Notice how after (nconc x '(4 5)) is evaluated, the value of x itself has changed.

Now, let's replicate this with our story themes.

First, re-initialize your lists in the *scratch* buffer to ensure you have a clean slate.

(setq themes-tech '("AI consciousness" "biopunk"))
(setq themes-social '("dystopian governance" "post-scarcity economics"))

This time, use nconc.
```
(nconc themes-tech themes-social)
```
The return value is the combined list, just as with append.
But now, inspect the original themes-tech variable.
```
themes-tech
```
When you evaluate this, you will see ("AI consciousness" "biopunk" "dystopian governance" "post-scarcity economics"). The original list has been destructively modified. It is no longer the list it once was. The themes-social list remains unchanged because it was the last argument.

3. Performance, Safety, and Side Effects

You might be asking: if nconc is so dangerous, why does it exist? The answer lies in the classic engineering trade-off: performance versus safety.

append: Safe and predictable. It creates copies, so it never has unexpected side effects. This safety comes at the cost of performance; copying a very long list can be slow and memory-intensive.
nconc: Fast and memory-efficient. It avoids creating copies, which is a significant advantage when concatenating large lists. This performance comes at the cost of safety; you can inadvertently modify a list that another part of your code was relying on.

The following resource provides an excellent overview of the concept of destructive functions in Lisp.

Lisp lists and destructive functions

This article from the 'Yoo Box' blog generalizes the concept of destructive vs. non-destructive functions, which will help solidify your understanding.

Please read the section titled '11. destructive functions and non-destructive functions'. Pay close attention to the table that pairs non-destructive functions (like append) with their destructive counterparts (like nconc). Also, note the explanation of 'Property P2'—that non-destructive functions do not modify their arguments. This is the key difference.

4. The Danger: Unintended Consequences with Shared Structure

The real danger of nconc appears when you modify a list that shares structure with another list. This can lead to baffling bugs where changing one variable mysteriously affects a completely different one.

Let's construct a scenario related to your work. Imagine you're managing content for a digital publication.

Define a list of evergreen article topics.

(setq evergreen-articles '("Guide to FOSS for Creatives" "History of Sci-Fi"))

Define a list of articles for this week's newsletter.

(setq weekly-articles '("New Elisp Features" "VimConf 2024 Highlights"))

You create a master plan by appending the evergreen list to the weekly list. Remember, append copies all but the last argument. This means master-plan will get a copy of weekly-articles, and its tail will point to the original evergreen-articles list.
```
(setq master-plan (append weekly-articles evergreen-articles))
```
At this point, master-plan is ("New Elisp Features" "VimConf 2024 Highlights" "Guide to FOSS for Creatives" "History of Sci-Fi").
Now, an editor decides to destructively add a new article to the evergreen-articles list.
```
(nconc evergreen-articles '("Interview with a Futurist"))
```
This changes evergreen-articles to ("Guide to FOSS for Creatives" "History of Sci-Fi" "Interview with a Futurist"). This seems fine.
But what happened to master-plan? Let's check.
```
master-plan
```
Evaluating this reveals ("New Elisp Features" "VimConf 2024 Highlights" "Guide to FOSS for Creatives" "History of Sci-Fi" "Interview with a Futurist"). Because master-plan shared the underlying structure of evergreen-articles, modifying the latter also modified the former. This "spooky action at a distance" is the primary reason destructive functions must be used with extreme care.

5. Best Practices for Using `nconc`

So, how do you use nconc safely?

Rule of Thumb: Only use a destructive function on a list that you "own" exclusively—that is, a list you've just created and are sure no other part of the program holds a reference to.
The Pitfall of Quoted Lists: Never use nconc on a quoted list literal, like (nconc '(a b) my-list). The Lisp system is free to treat all instances of '(a b) as pointers to the same constant data in memory. Modifying it once could permanently alter the literal for your entire Emacs session, leading to bizarre and hard-to-diagnose bugs. The GNU Manual, which you read earlier, correctly calls this a "common pitfall."
The Safe Idiom: If you need the performance of nconc but are operating on a list that might be shared, create an explicit copy first.
```
;; Unsafe - might modify a shared list
(nconc shared-list new-items)

;; Safe - operates on a fresh copy
(nconc (copy-list shared-list) new-items)
```
This gives you the best of both worlds: you avoid modifying the original shared list, but you also avoid the cost of copying new-items if it's very large.

Conclusion

You have now explored the critical difference between non-destructive and destructive list concatenation in Elisp. This understanding is a significant step forward in your journey to mastering the Lisp way of thinking.

Key Takeaways:

append is non-destructive. It returns a new list by copying its arguments (except the last one), leaving the originals untouched. It is safe but potentially slower.
nconc is destructive. It splices lists together by modifying the cdr of the last cons cell of the preceding list. It is fast and memory-efficient but can cause unexpected side effects if the list is shared.
The choice between them is a trade-off between safety and performance.
Use destructive functions only on data you are certain is not referenced elsewhere. When in doubt, use the safe, non-destructive version or operate on a copy.

Preview of the Next Lesson

We have now covered several ways to create, combine, and modify lists. The next logical step is to process the data held within them. In the upcoming lesson, we will cover the learning outcome: "Iterate over a list using 'dolist' and apply a function to each element using 'mapcar'." These are two of the most common and powerful tools for working with every element in a list, and they form the backbone of many Elisp programs.

Recommended resources