"Piping Operations with LCEL"

The Concept of Sequential Chains

In many real-world scenarios, a single LLM call is insufficient. You might need to perform a sequence of operations. For instance:

1. Generate a product description from key features.
2. Translate that description into multiple languages.
3. Generate social media posts from the description.

Each of these steps could be its own chain. A sequential chain is simply one that executes these steps in order, passing the output of the previous step as input to the next. Your experience with data engineering pipelines, where data flows through a series of transformation stages, provides a strong parallel. With LCEL, we define this flow declaratively.

The following article offers a great high-level view of different chain structures.

Building a Sequential Chain

Let's put this into practice. We'll build a chain that first generates a creative concept and then writes a short story based on that concept.

Open your src/lcel_quickstart/main.py file and modify it with the following code. We'll build it up step-by-step.

1. Define the Component Chains

First, let's define two separate chains. The first chain will generate a creative movie plot idea, and the second will write a short opening scene based on that plot.

import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableLambda # New import




# Load environment variables
load_dotenv()
if "OPENAI_API_KEY" not in os.environ:
    raise ValueError("OPENAI_API_KEY not found in .env file")

def main():
    """
    Demonstrates a sequential LLM chain using LCEL.
    """
    model = ChatOpenAI(model="gpt-4o-mini")
    parser = StrOutputParser()




    # Chain 1: Generates a creative plot idea
    prompt_idea = ChatPromptTemplate.from_template(
        "Generate a one-sentence creative plot idea for a movie about {topic}."
    )
    chain_idea = prompt_idea | model | parser




    # Chain 2: Writes an opening scene from a plot idea
    prompt_scene = ChatPromptTemplate.from_template(
        "Write a compelling, one-paragraph opening scene for a movie with the following plot:\n\n{plot_idea}"
    )
    chain_scene = prompt_scene | model | parser

We now have two independent, runnable chains: chain_idea and chain_scene.

2. Composing the Chains

How do we connect them? A naive attempt might be chain_idea | chain_scene. However, this will fail. chain_idea outputs a simple string, but chain_scene expects a dictionary with a plot_idea key (i.e., {"plot_idea": "..."}).

The flow of data must be explicit. The output of the first step must be correctly formatted as the input for the second. This is where the true compositional power of LCEL becomes apparent. One clean way to handle this is by inserting a custom function to bridge the two chains.

Integrating Custom Logic with RunnableLambda

LCEL provides RunnableLambda to wrap any Python function, making it a seamless component in a chain. This is incredibly useful for data transformation, custom logic, or any operation that doesn't involve an LLM call.

Let's define a function that takes the output of chain_idea (a string) and wraps it in the dictionary format expected by chain_scene.

    # ... inside main() function ...




    # A function to bridge the two chains
    def format_input_for_scene(idea: str) -> dict:
        """Takes the plot idea string and formats it as input for the next chain."""
        return {"plot_idea": idea}

Now, we can use RunnableLambda to insert this function into our sequence.

    # ... inside main() function ...




    # The complete sequential chain
    sequential_chain = (
        chain_idea
        | RunnableLambda(format_input_for_scene)
        | chain_scene
    )




    # Invoke the full chain
    print("--- Generating sequential chain response ---")
    response = sequential_chain.invoke({"topic": "a haunted particle accelerator"})
    print(response)

if __name__ == "__main__":
    main()

Run the script: uv run src/lcel_quickstart/main.py.

You will see a short movie scene generated from a plot idea that was itself generated in the first step of the chain. We have successfully created a two-step sequential LLM workflow. The RunnableLambda acts as the "glue," ensuring data compatibility between the components.

This video provides a great walkthrough of the core runnables, including RunnableLambda.

The Pinecone article "LangChain Expression Language Explained" also provides a concise code example for RunnableLambda that reinforces this concept.

The Inherent Benefits: Streaming and Batching for Free

This declarative, compositional approach unlocks powerful features automatically. The sequential_chain object we created isn't just a sequence of function calls; it's a first-class Runnable object. This means it automatically supports streaming, batching, and asynchronous operations with no additional implementation effort.

For example, to stream the final response as it's generated, you would simply change the last part of the script from .invoke() to .stream():

# To stream the response:
print("\n--- Streaming sequential chain response ---")
for chunk in sequential_chain.stream({"topic": "a library that contains every book never written"}):
    print(chunk, end="", flush=True)
print()

To process multiple topics in parallel, you would use .batch():

# To batch process multiple inputs:
print("\n--- Batch processing topics ---")
inputs = [
    {"topic": "a chef who can cook emotions"},
    {"topic": "an archaeologist who discovers time is a solid object"},
]
results = sequential_chain.batch(inputs)
for i, result in enumerate(results):
    print(f"\n--- Result for Topic {i+1} ---")
    print(result)

This is a significant advantage over a manual implementation, where you would need to write considerable boilerplate code to handle streaming and parallel execution.

The following video demonstrates these built-in capabilities.

Conclusion

In this lesson, we elevated our understanding of LCEL from simple linear chains to more powerful sequential compositions. You learned how to connect multiple chains together, with the output of one feeding the input of the next, a fundamental pattern for building complex applications.

Key takeaways include:

• Sequential Composition: LCEL allows you to pipe components together (A | B | C) to create multi-step workflows.
• Data Flow is Key: The output format of one component must match the input format of the next.
• RunnableLambda: This is your tool for inserting custom Python logic (like data transformation) into a chain, acting as flexible "glue" between components.
• Built-in Production Features: Chains built with LCEL automatically support advanced execution strategies like streaming and batching via the .stream() and .batch() methods, providing significant value over manually coded workflows.

We have now explored how data can flow sequentially through a chain. However, many applications, especially the RAG pipeline we aim to build, require more complex data flows. We need to run some operations in parallel and ensure the original user query is passed through multiple steps. In our next lesson, we will tackle exactly this by introducing RunnableParallel and RunnablePassthrough, the final core LCEL components you'll need.