Contextual Chat: Leveraging History for Better Prompts

1. The Key to Coherent Conversation: Context

An LLM, by itself, is stateless. To make a conversation coherent, we must provide the context of previous turns in every new request. There are a couple of ways to do this with Ollama, but the most common and flexible method is to use the /api/chat endpoint, which is specifically designed for conversational AI.

Unlike the /api/generate endpoint we used previously, /api/chat accepts a list of messages, where each message has a role (system, user, or assistant) and content.

To understand the Chat API better, please watch the first couple of minutes of this video.

Our plan is to adopt this chat-based approach. We will make our Rust backend stateful, meaning it will store the conversation history and manage the context for us.

2. Upgrading the Rust Backend for Chat

We need to make three key changes in our Rust code:

1. Define new structs to match the /api/chat JSON structure.
2. Create a shared, thread-safe place to store the conversation history.
3. Write a new function to call the /api/chat endpoint with the history.

This article provides a fantastic, complete example of a Rust chatbot using the same techniques we'll be implementing. It's a great reference for the code we're about to write.

Step 2.1: Defining the Chat Data Structures

First, let's add the necessary structs to your main.rs. These will use serde to serialize into the JSON format the /api/chat endpoint expects.

use serde::{Serialize, Deserialize};

// Represents a single message in the conversation history



#[derive(Serialize, Deserialize, Clone, Debug)]
struct Message {
    role: String,
    content: String,
}

// The request body for the /api/chat endpoint



#[derive(Serialize, Debug)]
struct ChatRequest {
    model: String,
    messages: Vec<Message>,
    stream: bool,
}

// Part of the streaming response from the /api/chat endpoint



#[derive(Deserialize, Debug)]
struct ChatResponse {
    message: Message,
    done: bool,
}

This structure is also detailed in another excellent tutorial, which you can refer to if you need more context on the struct definitions. The section "Mapping Ollama’s API to Rust structs" in the article "Building Doit Part 2" (resource ID LINK) provides a well-commented breakdown.

Step 2.2: Managing Conversation State

To store the history, we need a global, thread-safe variable. We'll use a Mutex to ensure that only one thread can access the history at a time, wrapped in a lazy_static macro to ensure it's initialized safely.

Add lazy_static and tokio::sync::Mutex to your Cargo.toml if they aren't there already.

[dependencies]



# ...
lazy_static = "1.4.0"
tokio = { version = "1", features = ["full"] }



# ...

Now, define the static history variable in main.rs.

use lazy_static::lazy_static;
use tokio::sync::Mutex;
use std::collections::Vec;

lazy_static! {
    // A globally accessible, thread-safe vector to store our conversation
    static ref CONVERSATION_HISTORY: Mutex<Vec<Message>> = Mutex::new(Vec::new());
}

Step 2.3: The New Chat Generation Logic

Let's adapt our old generate_dialogue function to work with the new Chat API. This new async function will handle the logic of sending the history and processing the streamed response.

async fn generate_chat_response() -> Result<String, Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();
    let history = CONVERSATION_HISTORY.lock().await; // Lock the history for reading

    let request_body = ChatRequest {
        model: "llama3".to_string(), // Or your preferred model
        messages: history.clone(),   // Send a clone of the current history
        stream: true,
    };
    
    // We're done reading, so we can drop the lock manually before the network call
    // This is good practice to avoid holding locks longer than necessary.
    drop(history);

    let mut res = client
        .post("http://localhost:11434/api/chat")
        .json(&request_body)
        .send()
        .await?;

    let mut full_response = String::new();
    while let Some(chunk) = res.chunk().await? {
        // We need to handle multiple JSON objects in a single chunk
        let chunk_str = String::from_utf8_lossy(&chunk);
        for line in chunk_str.lines() {
            if line.trim().is_empty() { continue; }
            if let Ok(response_part) = serde_json::from_str::<ChatResponse>(line) {
                full_response.push_str(&response_part.message.content);
                if response_part.done {
                    break;
                }
            }
        }
    }
    
    Ok(full_response)
}

3. Creating a Stateful FFI for Chat

With the core Rust logic in place, let's expose it to Unity through a new, stateful FFI. Instead of one function, we'll have two: one to start/reset the conversation and one to send a message.

Add these new FFI functions to your main.rs.

// FFI function to initialize or reset the conversation with a system prompt



#[no_mangle]
pub extern "C" fn init_conversation_ffi(system_prompt: *const c_char) {
    let prompt = unsafe { CStr::from_ptr(system_prompt) }.to_string_lossy().into_owned();

    tokio::spawn(async move {
        let mut history = CONVERSATION_HISTORY.lock().await;
        history.clear();
        history.push(Message {
            role: "system".to_string(),
            content: prompt,
        });
    });
}

// FFI function to send a user message and get an AI response



#[no_mangle]
pub extern "C" fn send_user_message_ffi(user_prompt: *const c_char, callback: DialogueCallback) {
    let prompt = unsafe { CStr::from_ptr(user_prompt) }.to_string_lossy().into_owned();

    tokio::spawn(async move {
        // Add the user's message to the history
        { // Scoped lock
            let mut history = CONVERSATION_HISTORY.lock().await;
            history.push(Message {
                role: "user".to_string(),
                content: prompt,
            });
        } // Lock is released here

        // Generate the response based on the updated history
        match generate_chat_response().await {
            Ok(dialogue) => {
                // Add the AI's response to the history
                { // Scoped lock
                    let mut history = CONVERSATION_HISTORY.lock().await;
                    history.push(Message {
                        role: "assistant".to_string(),
                        content: dialogue.clone(),
                    });
                } // Lock is released here
                
                // Send the response back to C#
                let c_dialogue = CString::new(dialogue).unwrap();
                callback(c_dialogue.as_ptr());
                std::mem::forget(c_dialogue);
            }
            Err(e) => {
                let error_message = format!("Error: {}", e);
                eprintln!("{}", error_message);
                let c_error = CString::new(error_message).unwrap();
                callback(c_error.as_ptr());
                std::mem::forget(c_error);
            }
        }
    });
}

Remember to recompile your Rust library (cargo build --release) and copy the updated file to your Unity project's Assets/Plugins folder.

4. Updating Unity to Have a Conversation

Now, let's hook up the C# side to our new FFI.

Step 4.1: Updating LLMService.cs

Open your LLMService.cs script and add the new function imports.

// Add these to your DllImport section
[DllImport(RustLib, CallingConvention = Cdecl)]
private static extern void init_conversation_ffi(string systemPrompt);

[DllImport(RustLib, CallingConvention = Cdecl)]
private static extern void send_user_message_ffi(string userPrompt, DialogueCallback callback);

// Add these new public methods for the game to call
public static void InitConversation(string systemPrompt)
{
    init_conversation_ffi(systemPrompt);
}

public static void SendUserMessage(string userPrompt)
{
    // The callback (ReceiveDialogue) is the same one we used in the previous lesson
    send_user_message_ffi(userPrompt, ReceiveDialogue);
}

Step 4.2: Updating ConversationUIController.cs

To see the conversation history, we'll modify the UI controller to append new messages instead of replacing the text.

// In ConversationUIController.cs
private void DisplayDialogue(string message)
{
    if (dialogueText != null)
    {
        // Append the new message, along with a role and new lines for readability
        dialogueText.text += $"AI: {message}\n\n";
    }
}

// Add a public method to add the player's message to the log
public void DisplayPlayerMessage(string message)
{
    if (dialogueText != null)
    {
        dialogueText.text += $"Player: {message}\n";
    }
}

Step 4.3: Testing the Conversation Flow

Finally, let's modify the DialogueTester.cs to test a multi-turn conversation. Instead of a text input field, we'll use different keys to send pre-written messages.

using UnityEngine;

public class DialogueTester : MonoBehaviour
{
    // Reference to the UI controller to show player's message
    public ConversationUIController uiController;

    private int messageIndex = 0;
    private readonly string[] playerMessages = {
        "Why is the sky blue?",
        "And what about during sunset?"
    };

    void Start()
    {
        Debug.Log("Initializing conversation...");
        string systemPrompt = "You are a helpful science teacher. Your answers should be concise and easy to understand for a 10-year-old.";
        LLMService.InitConversation(systemPrompt);
    }

    void Update()
    {
        // Use a single key to step through the conversation
        if (Input.GetKeyDown(KeyCode.T))
        {
            if (messageIndex < playerMessages.Length)
            {
                string message = playerMessages[messageIndex];
                Debug.Log($"Sending message: {message}");

                // Display the player's message in the UI
                if (uiController != null)
                {
                    uiController.DisplayPlayerMessage(message);
                }
                
                // Send the message to the Rust backend
                LLMService.SendUserMessage(message);

                messageIndex++;
            }
            else
            {
                Debug.Log("End of test conversation.");
            }
        }
    }
}

Assign your ConversationUIController to the public field on the DialogueTester script in the Unity Inspector.

Now, run the scene.

1. Press T. The first question, "Why is the sky blue?", will be sent. The AI should give a simple explanation.
2. Press T again. The second question, "And what about during sunset?", is sent. Because the Rust backend remembers the first question, the LLM will understand that "what about" refers to the color of the sky. It should give a relevant answer about sunsets, demonstrating that it has conversational memory.

Conclusion

Fantastic work! You have successfully upgraded our dialogue system from a simple one-shot generator to a stateful conversational agent. Our characters can now remember the context of a conversation, leading to much more believable and coherent interactions.

Key Takeaways:

• Chat API: Ollama's /api/chat endpoint, with its messages array, is the standard for building conversational experiences.
• Stateful Backend: By managing the conversation history (Vec<Message>) in Rust, we create a clean and powerful interface for Unity, keeping the core game logic in our Rust engine.
• Thread-Safe State: Using tools like Mutex is essential for safely managing shared data in a multi-threaded environment, which is standard when dealing with asynchronous FFI calls.

Preview of the Next Lesson:
We have now established the core foundation of our dynamic social simulation! We'll now pivot from character interactions to world interactions. In our next module, we will begin implementing the game's economy and a "buy mode," allowing players to purchase objects and furnish their homes.