LLM Function Calling for Agentic Workflows

Introduction

In our last lesson, we successfully built a Spring Boot application that communicates with an LLM using the Spring AI ChatClient. This gave your application the ability to answer questions based on the model's vast pre-existing knowledge. However, as we discussed, LLMs are fundamentally limited by their static training data—they have no access to real-time information, your company's private databases, or other external systems.

This lesson bridges that gap. We will explore LLM function calling, a powerful technique that allows your application to give a language model access to tools. You'll learn how this mechanism works and see how it forms the very core of creating the "agentic workflows" you're aiming to build. This is the conceptual leap from using an LLM as a simple chatbot to orchestrating it as an intelligent agent that can interact with the world to accomplish complex tasks.

Beyond Text Generation: Giving the LLM Tools

An AI agent, in essence, is a system that can perceive its environment, reason about a goal, and take actions to achieve it. A standalone LLM can reason, but it can't perceive or act beyond generating text. To make it a true agent, we need to equip it with tools.

In the context of software, these tools are simply functions—your Java methods, API clients, or database queries. Function calling is the mechanism by which an LLM can request the execution of one of these tools.

To begin, let's establish a clear definition of AI agents and their components. The article "What Are Agentic Workflows?" provides an excellent overview.

The most crucial insight from this reading is that the LLM does not execute the function itself. Instead, it recognizes when a tool is needed and generates a structured message (typically JSON) instructing your application on which function to call and what arguments to use. Your application code is always in control of the actual execution.

This point is often a source of confusion, but it's fundamental to understanding the security and architecture of these systems. The LLM suggests; the application executes.

The Agentic Workflow in Action

Let's break down the step-by-step process of an agentic workflow that uses function calling. Imagine a user asks your Spring Boot application: "What's the current value of my stock portfolio?" Your application can't answer this directly, and neither can the LLM. It requires combining private data (your portfolio) with real-time data (stock prices).

This is a perfect use case for function calling. The following diagram illustrates the complete, multi-step interaction.

Let's walk through this flow, which is the heart of how these systems operate:

1. User Asks a Question: The user sends the query, "What's the current value of my stock portfolio?" to your Spring Boot application.
2. App Sends Prompt + Functions: Your app sends a request to the LLM. This request includes not only the user's question but also definitions of the tools (Java functions) you've made available. For instance, you might provide two functions: getPortfolioShares() and getLatestStockPrice(String stockSymbol). You provide a name and a description for each, which the LLM uses to understand their purpose.
3. LLM Decides to Call a Function: The LLM analyzes the prompt. It understands that to calculate the portfolio value, it first needs to know which shares the user owns. It decides to call the getPortfolioShares() function.
4. App Executes the Function: The LLM responds to your app with a structured JSON object, essentially saying, "Please call the getPortfolioShares function for me with no arguments." Your Spring application receives this, finds the corresponding Java bean, and executes it. This method queries your database and returns a list of stocks, say [100 shares of AAPL, 300 shares of AMZN].
5. App Retrieves More Data: At this point, the initial goal is not yet met. The application, or more advanced agent logic, recognizes it needs the price for each stock. It might loop through the list and trigger more function calls.
6. App Sends Data Back to LLM: Your application makes another call to the LLM. This time, it says, "Here are the results of the functions you asked me to call: the user owns 100 shares of AAPL, and the latest price for AAPL is $170. The user owns 300 shares of AMZN, and the latest price for AMZN is $180."
7. LLM Generates the Final Answer: Now, the LLM has all the necessary information: the user's goal, the portfolio composition, and the real-time prices. It performs the final reasoning and calculation, then generates a natural language response.
8. App Returns the Answer: The LLM sends back the final answer: "The current value of your portfolio is $71,000 (100 shares of AAPL at $170/share and 300 shares of AMZN at $180/share)." Your application forwards this to the user.

This multi-step "conversation" between your application and the LLM is the essence of an agentic workflow. The video "LLM Function Calling - AI Tools Deep Dive" by Adam Lucek provides an excellent visual explanation of this entire sequence.

Implementing Functions in Spring AI

So, how do you provide those function definitions to the LLM in a Spring Boot application? Given your background, you'll appreciate that Spring AI makes this quite idiomatic. You define your functions as standard Java beans, typically implementing the java.util.function.Function or java.util.function.Supplier interfaces.

The key is providing a description via an annotation (@Description). This natural language description is what the LLM uses to understand what the function does and when to call it.

Piotr's TechBlog has an article that walks through this exact process, building the portfolio value example we just discussed.

By defining functions as described, you empower the ChatClient to automatically handle the multi-step workflow. When the LLM responds with a function call request, Spring AI intercepts it, executes the correct bean in your application context, and sends the result back to the LLM, abstracting away much of the boilerplate logic.

From a Simple Loop to an Intelligent Agent

This ability to use tools is what elevates a simple LLM call into an agentic system. An AI agent is more than just a single function call; it's a system built around a reasoning loop.

As shown in the architecture diagram above, a mature agent integrates:

• LLM Core: The "brain" that performs reasoning and decision-making.
• Tool-Use: The function-calling capability we've been discussing, allowing interaction with the outside world.
• Memory:
  • Short-term memory holds the context of the current conversation (like the intermediate function call results).
  • Long-term memory stores information across sessions, allowing the agent to learn and personalize.
• Specialized Agents: For complex problems, a primary agent might delegate tasks to other agents with specific roles, such as a Planning Agent to break down a problem or a Reflection Agent to critique a result and suggest improvements.

This modular structure is how you can move from writing a simple script to designing a robust, autonomous system that can solve business problems.

Your Turn: Conceptual Application

Before our next lesson where we'll diagram a workflow, take a moment to apply this concept to your own work environment.

Think of a recurring task or question you handle that requires fetching data before you can provide an answer. This could be anything from checking a feature flag's status in a configuration service to pulling user details from a database.

1. Define the Task: Briefly describe the task.
2. Design the Function: Describe this task as a Java function that an LLM could call.
   • What would be a good, descriptive function name?
   • What clear, concise description would you give it so an LLM knows when to use it?
   • What are the necessary input parameters (and their types)?
   • What information would the function return (and what would be its type)?

Example:

• Task: Check the deployment status of a service in our staging environment.
• Function Design:
  • Name: getDeploymentStatus
  • Description: "Retrieves the current deployment status (e.g., 'SUCCESS', 'IN_PROGRESS', 'FAILED') and version of a given service in the staging environment."
  • Parameters: (String serviceName)
  • Return Value: DeploymentInfo (a record or class containing String status and String version).

Completing this thought exercise will set a strong foundation for our next lesson.

Conclusion

In this lesson, we moved beyond simple LLM prompts and into the world of building intelligent agents. You've learned that LLM function calling is the critical mechanism that allows a language model to interact with external tools and data, breaking free from the confines of its static training.

Here are the key takeaways:

• Function calling gives LLMs tools: It allows them to access real-time data, private APIs, and databases.
• The LLM suggests, your app executes: The model's output is a JSON request to run a function, which your application code is responsible for executing securely.
• It's a multi-step workflow: A single user query often results in a back-and-forth conversation between your app and the LLM to gather data before a final answer is synthesized.
• Function calling is the foundation of agentic workflows: This tool-use capability, combined with reasoning and memory, is what defines an AI agent.

You now have the conceptual framework to understand how to build applications where the AI doesn't just answer questions, but actively takes steps to find the answer. In our next lesson, we will put this knowledge into practice by diagramming a simple agentic workflow, translating these abstract steps into a concrete architectural plan.