Atomic Redis Transactions with Lua

Your learning outcome for this lesson:

By the end of this lesson, you will be able to write a basic Lua script to execute multiple Redis commands atomically. This practical skill is the cornerstone of building correct and performant distributed algorithms, which we will do in the next module.

1. How Redis Executes Lua Scripts

When you send a Lua script to Redis, the server guarantees its atomic execution. This means Redis will not process any other command from any other client from the moment the script begins until it ends. The entire script is treated as a single, indivisible operation.

This diagram perfectly illustrates the concept.

This atomic guarantee is why Lua is the solution to the read-modify-write problem. It allows us to package our logic—the "modify" step—with our reads and writes into a single, safe transaction.

In addition to atomicity, running logic server-side has another major benefit: it reduces network latency. Instead of your application sending multiple commands back and forth to Redis (e.g., GET, then SET), it sends one script, and all the intermediate operations happen within the Redis server itself.

2. The Anatomy of a Redis Lua Script

To execute a script, you use the EVAL command. Its basic structure is:

EVAL script numkeys key [key ...] arg [arg ...]

• script: The string containing your Lua code.
• numkeys: The number of Redis keys you are passing to the script.
• key [key ...]: The names of the keys. Inside the script, these are accessed through a special table called KEYS. For example, the first key is KEYS[1].
• arg [arg ...]: Any additional arguments that are not keys (e.g., values, limits, timestamps). These are accessed through a table called ARGV. For example, ARGV[1].

Separating keys from other arguments is crucial for Redis, especially in a clustered environment, as it helps Redis know which node the script should run on.

Within the script, you interact with Redis data using the redis.call() function. For example, redis.call('INCR', KEYS[1]) executes the INCR command on the first key passed to the script.

To familiarize yourself with the basic syntax, please review the following material.

3. Writing Your First Atomic Script

Let's apply this knowledge to solve the problem from our last lesson: the non-atomic INCR and EXPIRE sequence. We need a script that increments a counter and sets its expiration time, but only on the first increment. This ensures we don't accidentally overwrite the TTL of an existing key on subsequent increments.

Here is the Lua script that accomplishes this atomically:

-- KEYS[1] is the rate limit counter key, e.g., "ratelimit:ip:127.0.0.1"
-- ARGV[1] is the expiration time in seconds, e.g., 60

-- Increment the key and get the new value.
-- If the key does not exist, Redis creates it with a value of 1.
local current_count = redis.call('INCR', KEYS[1])

-- Check if this was the first increment (the value will be 1).
if current_count == 1 then
  -- If so, set the expiration time.
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end

-- Return the current count to the application.
return current_count

Let's break this down:

1. We execute INCR on the key provided in KEYS[1]. The result is stored in a local variable current_count.
2. We use a simple if statement to check if current_count is 1. This condition is only true when the key was just created by the INCR command.
3. If the condition is true, we call EXPIRE on the same key, using the expiry value from ARGV[1].
4. Finally, we return the current_count. The value returned by a Lua script becomes the return value of the EVAL command in your application.

This entire block of logic runs as one atomic operation, completely eliminating the race condition where an application could crash between INCR and EXPIRE.

4. Executing the Script from Spring Boot

Now, let's see how you would execute this from your Java application using Spring Data Redis, which you've already configured. The standard approach is to use RedisTemplate and a RedisScript object.

First, you would typically load your Lua script from a file, but for simplicity, we can define it as a string.

import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.data.redis.core.script.RedisScript;
import java.util.Collections;

public class AtomicCounter {

    private final RedisTemplate<String, String> redisTemplate;
    private final RedisScript<Long> script;

    public AtomicCounter(RedisTemplate<String, String> redisTemplate) {
        this.redisTemplate = redisTemplate;
        
        // Define the Lua script text
        String luaScript = "local current_count = redis.call('INCR', KEYS[1])\n"
                         + "if current_count == 1 then\n"
                         + "  redis.call('EXPIRE', KEYS[1], ARGV[1])\n"
                         + "end\n"
                         + "return current_count";
                         
        // Create a RedisScript object. Specify the return type.
        this.script = new DefaultRedisScript<>(luaScript, Long.class);
    }

    public Long incrementAndExpire(String key, long expirySeconds) {
        // Execute the script.
        // The first argument is the RedisScript object.
        // The second is a List of keys (KEYS table).
        // The third and subsequent are the arguments (ARGV table).
        return redisTemplate.execute(
            script,
            Collections.singletonList(key),
            String.valueOf(expirySeconds)
        );
    }
}

In this example:

• We create a DefaultRedisScript, telling Spring Data Redis that we expect the script to return a Long.
• In the incrementAndExpire method, we call redisTemplate.execute().
• We pass the script object, a list containing our single key, and the expiry time as an argument.
• Spring Data Redis handles sending the EVAL command to Redis and deserializing the numeric response back into a Java Long.

5. Best Practices for Writing Lua Scripts

As you start writing more complex scripts for your rate limiter, it's wise to follow some established best practices.

One other important practice mentioned is script caching. Repeatedly sending the same large script string with EVAL is inefficient. Redis allows you to load a script once with SCRIPT LOAD, which returns an SHA1 hash. You can then execute the script using this hash with the EVALSHA command. This is more performant and is the standard for production systems. We will incorporate this optimization later in the course.

Conclusion

Congratulations! You've successfully bridged the gap from theory to practice. You now understand not only why atomic operations are necessary but also how to implement them using Lua in Redis.

Key Takeaways:

• Lua scripts run atomically on the Redis server, preventing any other command from interfering.
• Scripts are executed with the EVAL command, which takes the script text, a count of keys, the keys themselves (KEYS table), and other arguments (ARGV table).
• From within Lua, you use redis.call() to execute standard Redis commands.
• In a Spring Boot application, you use RedisTemplate and RedisScript objects to execute Lua scripts in a clean, type-safe way.
• Following best practices, like using KEYS/ARGV correctly and keeping scripts focused, leads to more robust and maintainable code.

Next Steps

Now that you have the essential tool—atomic scripting—we can begin designing our distributed rate limiting algorithms. In the next lesson, "Design a Redis key structure for the sliding window and token bucket algorithms," we will plan the data models we need. Before writing the complex Lua logic, we first need to decide precisely what data we'll store in Redis and how we'll organize the keys for efficient access.