LLM, Code, or SaaS: Choosing the Right Solution

Welcome back! In our previous lesson, we mapped out the major product patterns in the GenAI landscape, from chatbots to autonomous agents. You now have a catalog of what you can build. Today, we tackle an even more fundamental question for a product builder: for any given problem, should you use an AI model at all?

This might be the most important strategic decision you make. As an indie hacker, your resources—time and money—are finite. Choosing the right tool for the job is paramount. Using a powerful, expensive LLM for a task that a simple script could handle is like using a sledgehammer to crack a nut. It's not just inefficient; it's a liability.

This lesson will provide you with a mental framework to decide when a feature warrants an LLM, and when you should rely on deterministic code or an existing SaaS API. By the end, you'll be able to dissect a problem and choose the most effective, reliable, and cost-efficient implementation path.

The Most Expensive Operation in Modern Software

Let's begin with a strong, and perhaps surprising, assertion that should be top-of-mind for any product builder.

The key takeaway here is a powerful guiding principle: "Most effective 'AI agents' aren't actually that agentic at all. They're mostly deterministic software with strategic LLM calls placed exactly where they add value."

Your default mindset should not be "How can I solve this with an LLM?" but rather "How can I solve this with the simplest, most reliable code possible?" Only when a specific component of the problem is impossible or impractical to solve deterministically should you consider making that "expensive and dangerous" API call.

This leads to a simple, three-step development process:

1. Break your product idea down into its fundamental components.
2. Try to solve each component with standard software engineering (if-then logic, loops, database queries, calling a traditional API).
3. Only for the components that resist this approach—the ones that require reasoning, understanding fuzzy language, or generating novel content—should you introduce an LLM.

Let's see how this plays out in a common product category: the chatbot.

The Chatbot Dilemma: Rules vs. Reasoning

The term "chatbot" can describe two very different systems. The choice between them is a perfect case study for our decision framework.

The video makes the trade-off crystal clear.

• For predictable, factual queries, like a customer asking "What are your store hours?" or "What is your return policy?", a rule-based system is superior. It's cheaper, faster, and 100% reliable. You don't want your support bot "creatively" hallucinating your store's opening times.
• For creative or open-ended tasks, like "Help me brainstorm ideas for a fantasy novel" or "Summarize this long and messy conversation," a generative AI chatbot is the only viable option. Its ability to understand nuance and generate novel text is essential.

This isn't just about chatbots. This principle applies to any feature you build.

A Practical Decision Framework

When you're evaluating a product idea or feature, ask yourself the following questions. The answers will guide you to the right tool.

1. Is the task deterministic or probabilistic?

   • Deterministic: The correct output can be defined by a set of rules, no matter how complex. Calculating taxes, validating a user's password, or checking if a username is taken are all deterministic. Use code.
   • Probabilistic: The task involves ambiguity, creativity, or understanding natural human language. Summarizing a document, judging the sentiment of a review, or writing a marketing email are probabilistic. Consider an LLM.

2. What is the tolerance for error and non-determinism?

   • Low Tolerance: The output must be factually correct and consistent every single time. Processing a financial transaction, controlling a physical system, or returning a user's account balance has no room for error. Use code.
   • High Tolerance: The output is a draft, a suggestion, or a creative starting point. The user is in the loop and can easily verify or edit the output. Generating a blog post outline or a draft reply to an email can tolerate some imperfection. An LLM is a good fit.

3. Is the core problem about understanding or execution?

   • Execution: The inputs are already structured, and the task is to perform a series of well-defined steps. For example, "Take this user ID, find their subscription in the database, and check if the endDate is in the past." This is just database logic. Use code.
   • Understanding: The inputs are unstructured and messy, like a user's email or a voice command. The core challenge is to figure out what the user wants. For example, interpreting "I bought this a couple weeks ago but I'm not happy with it, can I get my money back?" into a structured intent: refund_request is a task of understanding. LLMs excel at this.

4. Have you checked for a regular API first?

   • Before you build an LLM-powered system to find the best flight from JFK to SFO, remember that services like the Google Flights API or Skyscanner API already exist. Don't reinvent the wheel. If a traditional SaaS company offers a reliable, deterministic API for your problem, it will almost always be cheaper and more stable than building your own LLM-based solution.

The Two Lifecycles

This decision framework also helps explain why AI products feel different to build and maintain. You are essentially managing two different types of development lifecycles within one product.

When you choose to use deterministic code, you operate in the world on the left: Discover, Design, Develop, Test, Release, Maintain. It's a well-trodden path.

When you introduce an LLM, you are embracing the world on the right for that component. Notice the iterative loops between "Problem Definition," "Data Acquisition," "Model Fine-Tuning," and "Testing." This reflects the experimental nature of working with AI. You don't just "code" the right prompt; you experiment, test, monitor, and iterate constantly.

The goal of a savvy product builder is to keep as much of your system as possible in the clean, predictable cycle on the left, and to surgically introduce the iterative cycle on the right only for the parts of your product that absolutely need that probabilistic magic.

Conclusion

You've now moved from classifying AI patterns to making critical strategic decisions about their use. Knowing when not to use an LLM is a skill that will save you countless hours of development and debugging, and will result in more robust and reliable products.

Key Takeaways:

• An LLM API call is a slow, expensive, and non-deterministic operation. Use it sparingly.
• The most successful AI products are primarily deterministic software with strategic, well-placed LLM calls.
• Your default should always be to use deterministic code or a standard API. Only use an LLM when the task is inherently probabilistic, requires understanding fuzzy inputs, or involves creative generation.
• Use the decision framework: Is it deterministic? What's the error tolerance? Is it about understanding or execution? Is there an API for that?
• By minimizing your use of LLMs, you minimize the parts of your product that are subject to the complex, iterative, and less predictable AI development lifecycle.

This lesson concludes the first module of our course. We've established a high-level understanding of what LLMs are, what they can do, and—most importantly—when you should (and shouldn't) use them. In our next module, we'll roll up our sleeves, move from strategy to practice, and get our hands on the code. We'll start by setting up a Python environment and making our very first API call to an LLM.