Interpreting Iceberg Snapshot Records and Table History

Hello again. Last time, you distinguished the table metadata JSON, manifest lists, manifests, and stored data or delete files. You also saw that the JSON contains snapshot records, while each record points to a separate manifest list.

This lesson looks inside one of those records. By the end, you should be able to read its fields as a description of one immutable table state, place it in table history, and understand why a backup must not treat “the current snapshot” as a fixed identifier.

A snapshot is a state, not a copy

A snapshot does not contain another copy of all the table’s rows. It identifies a state of the table through a manifest list, whose manifests identify the files relevant to that state. Two snapshots can reuse many of the same manifests and data files while differing in others.

This short video gives a visual account of that idea. It also compares a snapshot to a Git commit: the comparison is useful for thinking about identifiable historical states, though an Iceberg snapshot points to stored file metadata rather than containing a self-sufficient copy of the table.

There are two different choices involved in loading a table. HMS’s metadata_location selects a table metadata JSON version. Within that version, a current-snapshot ID selects which snapshot is current for ordinary reads. The JSON may also retain records for earlier snapshots. Thus, “which metadata version did I read?” and “which snapshot did I select?” are related but different questions.

Reading the snapshot record

The Iceberg specification provides the authoritative field definitions. Read the small portions below before examining the example; the important distinction is between fields that identify or locate a state and fields that describe the change that produced it.

Here is an illustrative excerpt, not a JSON file from your service. Imagine that one metadata JSON version records these two snapshots and sets current-snapshot-id to 8102:

{
  "current-snapshot-id": 8102,
  "snapshots": [
    {
      "snapshot-id": 8101,
      "sequence-number": 1,
      "timestamp-ms": 1735689600000,
      "manifest-list": "s3://lake/orders/metadata/list-8101.avro",
      "summary": {
        "operation": "append",
        "added-data-files": "2"
      }
    },
    {
      "snapshot-id": 8102,
      "parent-snapshot-id": 8101,
      "sequence-number": 2,
      "timestamp-ms": 1735776000000,
      "manifest-list": "s3://lake/orders/metadata/list-8102.avro",
      "summary": {
        "operation": "append",
        "added-data-files": "1"
      }
    }
  ]
}

Read the second record in two parts. Snapshot 8102 is a distinct state whose parent is 8101; its manifest-list path tells you where to begin discovering the files for state 8102. The append operation and added-data-files count instead summarize what changed when 8102 was produced. They are not an inventory of everything present in 8102.

There is a format-version detail worth recognizing in unfamiliar code: a version 1 snapshot may use a manifests list of manifest locations instead of manifest-list. If manifest-list is present, that alternative must be omitted. You need not build support for either form without checking your tables; the point is not to assume a missing field always means a corrupt snapshot.

operation is inside summary, not a separate top-level field in the snapshot record. The specification names four common values:

• append: data files were added without removing files.
• replace: files changed without changing the table’s logical data, as in compaction.
• overwrite: a logical overwrite added and removed files.
• delete: data was logically removed and/or row-level delete files were added.

These describe the kind of change. For example, replace does not mean “there is no new snapshot,” and delete does not, by itself, mean an S3 object has already been deleted. Nor can a backup obtain the complete set of required paths merely by inspecting operation and summary counts.

Parent links are history, not file deltas

It is tempting to read parent-snapshot-id as an instruction to reconstruct a snapshot by starting with its parent and applying the summary’s changes. That is not how the snapshot record is used to locate its files. Each snapshot identifies its own manifest list. That list can include reused manifests alongside new ones, providing the route to the snapshot’s state. The parent ID describes the historical relationship between states.

History need not be a single uninterrupted chain of snapshots that all remain current ancestors. Iceberg’s query documentation shows an example in which a commit was rolled back: two snapshots share a parent, but only one lies on the current line of ancestry. The .snapshots metadata table is useful for inspecting snapshot records; .history shows when snapshots were made current.

To picture the rolled-back case, suppose S11 and S12 both name S10 as parent. A table can have made S11 current, rolled back to S10, and later made S12 current. In that situation, S11 remains an identifiable snapshot while retained, but it is not an ancestor of the current state S12. Neither a larger-looking ID nor mere presence in a snapshots list tells you which state is current. Consult the current-snapshot selection in the metadata version you actually loaded.

There is a similar distinction between times. A snapshot’s timestamp-ms describes its creation. History’s made_current_at describes when it became current; these need not be interchangeable, particularly when an older snapshot is selected again. Sequence numbers order successful snapshot-producing commits, but a metadata-only change can produce a new metadata JSON version without producing a new snapshot. Metadata version, snapshot ID, and sequence number answer different questions.

What this means when reading the backup code

Suppose a backup reads metadata JSON version V9, finds current snapshot 8102, and reads that record’s manifest-list location. If another writer later commits a new snapshot 8103, a later lookup of HMS’s metadata_location may select V10, where 8103 is current. That does not edit snapshot 8102 or its manifest list. It does mean that code which asks for “current” again may receive a different snapshot ID from the one it started with.

For now, the repository-reading rule is narrow: when you see a deserialized Snapshot, locate its snapshot-id and manifest-list, and determine which table metadata version supplied that record. Do not infer its full file set from its parent or summary. Later lessons will cover precisely how to interpret manifest entries, and then how concurrent commits affect a backup that rereads changing pointers.

Finally, immutability is not a retention guarantee. A snapshot record and the objects it references are not edited into a different state, but snapshot expiration and cleanup can eventually make an old state unavailable. That is a separate concern from identifying the right snapshot in the first place.

Key takeaways

A snapshot record identifies one table state through its ID and manifest-list location. Its parent ID places it in history; its sequence number and timestamp help describe ordering and creation time; its summary.operation and other summary entries describe the change that produced it, not the entire file inventory. The metadata JSON selects a current snapshot, but a later JSON version may select another without altering the earlier snapshot.

Next, you will follow a pinned snapshot’s manifest list and manifests to decide which files are actually visible in that state—including why an entry marked DELETED does not imply immediate removal from S3.