Thought-on-Thought: Prompting for Multiple Reasoning Paths

1. A Manual ToT Workflow: Human-in-the-Loop Search

At its heart, ToT is an algorithm. But before we automate it, it's incredibly instructive to walk through the steps manually. This approach, where you act as the "search algorithm," guiding the LLM through the generation and evaluation phases, reveals the core mechanics of the framework.

A great way to see this in action is with open-ended, qualitative problems where the "thoughts" are ideas or strategies rather than mathematical steps.

As the video demonstrates, you can orchestrate a ToT process using a sequence of distinct prompts. Let's formalize this pattern:

The Four-Phase Prompting Pattern

1. Brainstorming (Thought Generation):

   • Goal: Generate a diverse set of initial candidate solutions or paths.
   • Prompt Structure: You ask the model to generate multiple distinct solutions, often encouraging variety by specifying different factors to consider.
   • Example from video: "Could you brainstorm three distinct solutions? Please consider a variety of factors such as my background, timing, preparation..."

2. Evaluation (State Evaluation):

   • Goal: Assess the quality and viability of each generated thought.
   • Prompt Structure: You instruct the model to act as a critic, analyzing the pros, cons, potential challenges, and probability of success for each of the brainstormed solutions.
   • Example from video: "For each of the three proposed solutions, evaluate their potential. Consider the pros and cons... Assign a probability of success..."

3. Expansion (Lookahead/Refinement):

   • Goal: Deepen the analysis of each promising thought by exploring implementation details and potential obstacles.
   • Prompt Structure: For each solution, you ask the model to generate potential scenarios, strategies, and ways to handle unexpected outcomes.
   • Example from video: "For each solution, deep in the thought process, generate potential scenarios, strategies for implementation..."

4. Decision (Selection/Pruning):

   • Goal: Synthesize all the analysis to rank the solutions and make a final, justified recommendation.
   • Prompt Structure: You ask the model to rank the solutions based on the previous evaluation and expansion, providing a final justification.
   • Example from video: "Based on the evaluations and scenarios, rank the solutions in order of promise. Provide a justification..."

This manual workflow is a powerful pattern for tackling complex, ill-defined problems. You, the user, act as the search algorithm, deciding which thoughts to expand and when to move to the final decision.

2. Towards Automation: Specialized Prompts for Structured Problems

The manual workflow is excellent for exploration, but for more constrained problems like the Game of 24, we can design highly specialized prompts to automate the Generate-Evaluate-Select loop. Your computer science background will be particularly useful in seeing how these prompts are essentially function calls in a larger search algorithm.

Let's examine the prompt engineering used to automate ToT for the Game of 24.

Let's break down the key implementation patterns from this resource.

Step 2a: The "Propose" Prompt (Constrained Thought Generation)

Instead of a broad "brainstorm," this approach uses a one-shot prompt to generate a list of concrete, valid next steps.

Example Propose Prompt:

Input: 2 8 8 14
Possible next steps:
2 + 8 = 10 (left: 8 10 14)
8 / 2 = 4 (left: 4 8 14)
...
Input: 1 1 1 8
Possible next steps:

When given to an LLM, this prompt elicits a list of potential operations, such as 1 + 1 = 2 (left: 1 2 8), which become the child nodes in our search tree. This is a far more structured form of thought generation than the open-ended brainstorming we saw earlier.

Step 2b: The "Value" Prompt (Heuristic State Evaluation)

This is the most critical part of automating ToT. We use a separate LLM call with a specifically designed few-shot prompt to act as a heuristic function, evaluating the "promise" of each generated thought.

Example Value Prompt (for intermediate thoughts):

Evaluate if given numbers can reach 24 (sure/likely/impossible)
10 14 -> 10 + 14 = 24 -> sure
11 12 -> ... -> impossible
...
3 8 ->

The LLM's response (sure, likely, or impossible) is then programmatically mapped to a numerical value (e.g., sure=20, likely=1, impossible=0.001). This score allows the search algorithm to quantitatively compare different branches of the tree and prune the unpromising ones.

Step 2c: The Search Logic

With these generation and evaluation prompts, you can write a simple program to execute the search. The logic for one step of a Breadth-First Search (BFS) would look like this in pseudocode:

# Assume 'queue' holds the most promising nodes from the previous level
new_queue = []
for node in queue:
    # 1. GENERATE: Use the "propose" prompt with the numbers in 'node'
    candidate_thoughts = generate_thoughts(node.state)

    # 2. EVALUATE: For each candidate thought...
    for thought in candidate_thoughts:
        # Use the "value" prompt to get a score
        score = evaluate_thought(thought)
        
        # Create a new node with the thought and its score
        new_node = TreeNode(state=thought, value=score)
        new_queue.append(new_node)

# 3. SELECT/PRUNE: Sort 'new_queue' by score and keep the top N (the breadth_limit)
queue = sorted(new_queue, key=lambda x: x.value, reverse=True)[:breadth_limit]

This loop—Generate, Evaluate, Select—is the fundamental pattern of an automated ToT implementation. It uses targeted LLM calls to explore a problem space much more effectively than a single, monolithic prompt.

This console output from a real ToT implementation shows the iterative nature of the search, exploring and refining paths until a solution is found.

Conclusion

Today, you've bridged the gap between the theory of ToT and its practical implementation. You've seen that at its core, ToT is a pattern of generating multiple potential next steps and then evaluating them to intelligently guide a search toward a solution.

Key Takeaways:

• ToT can be implemented manually: A sequence of prompts for brainstorming, evaluating, and deciding allows a human to guide the search process, which is effective for complex, open-ended problems.
• Automation relies on specialized prompts: For structured problems, you can automate ToT using a "propose" prompt for generation and a "value" prompt for evaluation.
• The LLM acts as a heuristic function: The "value" prompt is a powerful technique that turns the LLM into a programmable heuristic for your search algorithm.
• The core loop is Generate-Evaluate-Select: Whether manual or automated, the fundamental pattern involves creating candidate thoughts, scoring them, and selecting the most promising ones to expand upon.

What's Next?

In this lesson, you learned how ToT uses a deliberate search to find the best reasoning path. In our next lesson, "Design self-consistency prompting strategies," we will explore a different, complementary technique for improving reliability. While ToT is a deep, evaluative search, self-consistency is a broader, simpler approach: it generates multiple reasoning paths independently and takes a "majority vote" on the final answer. Understanding both will give you a powerful and flexible toolkit for enhancing LLM performance.