Crafting LLM Dialogue for Interactive Conversations

1. Recap and The Next Step: From Prompt to API Call

In the previous lesson, we created a construct_prompt function that produced a detailed prompt like this:

You are Alex. Your personality traits are: Honesty-Humility is high...
You are currently feeling ecstatic. You are talking to Bob, who is a good friend.
You are performing the social interaction: 'Friendly Greeting'. Based on your personality and mood, generate a single, short line of dialogue...

Now, we need to send this to the LLM. While we could use a simple "generate" endpoint, a more robust approach for character interaction is to use a chat endpoint. Chat APIs are designed to understand conversational roles (system, user, assistant), which maps perfectly to our needs. We can put the character's persona and the situation into a system message to set the context, and the immediate instruction into a user message.

2. The Anatomy of an Ollama Chat Request

To communicate with Ollama's /api/chat endpoint, we need to send a JSON object that contains the model name and a list of messages. Each message has a role and content.

To see how this works in practice and how to handle the response, this blog post provides an excellent, detailed walkthrough.

As explained in the article, we need corresponding Rust structs to serialize our request and deserialize the response. Let's add them to our project.

Open your main.rs file. First, ensure you have serde and serde_json in your Cargo.toml with the derive feature enabled:

[dependencies]



# ... other dependencies
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
reqwest = { version = "0.12", features = ["json"] }
tokio = { version = "1", features = ["full"] }
anyhow = "1.0"

Now, add these structs to your main.rs. These are inspired by the resource you just read.

use serde::{Deserialize, Serialize};

// Represents a single message in the conversation history



#[derive(Serialize, Deserialize, Clone, Debug)]
struct Message {
    role: String,
    content: String,
}

// The request body for Ollama's /api/chat endpoint



#[derive(Serialize, Debug)]
struct ChatRequest {
    model: String,
    messages: Vec<Message>,
    stream: bool,
}

// The structure of a single chunk from the streamed response



#[derive(Deserialize, Debug)]
struct ChatResponse {
    message: Message,
    done: bool,
}

3. Implementing the Dialogue Generation Function

Now we'll replace our old llm_completion function with a more sophisticated one that uses the Chat API and handles streaming. This function will take our detailed prompt, split it into system and user messages, and manage the API call.

The logic for handling streamed responses can be tricky because the data arrives in chunks (NDJSON, or Newline Delimited JSON). The Building Doit article explained one way to handle this with a buffer. For a visual, live-coded explanation of the same process, the following video is very helpful.

Let's write our new function. Add this to main.rs. Notice how we split the detailed prompt from our previous lesson into a system message (the persona and situation) and a user message (the specific instruction).

use anyhow::Result;
use futures::StreamExt;
use std::io::{stdout, Write};

async fn generate_dialogue(
    system_prompt: &str,
    user_prompt: &str,
) -> Result<String> {
    let client = reqwest::Client::new();

    let request_body = ChatRequest {
        model: "llama3".to_string(), // Or your preferred model
        messages: vec![
            Message {
                role: "system".to_string(),
                content: system_prompt.to_string(),
            },
            Message {
                role: "user".to_string(),
                content: user_prompt.to_string(),
            },
        ],
        stream: true,
    };

    let mut stream = client
        .post("http://localhost:11434/api/chat")
        .json(&request_body)
        .send()
        .await?
        .bytes_stream();

    let mut full_response = String::new();
    let mut buffer = String::new();

    print!("\n[LLM Is Thinking]: ");
    stdout().flush()?; // Ensure the prompt is shown immediately

    while let Some(chunk_result) = stream.next().await {
        let chunk = chunk_result?;
        let text = String::from_utf8_lossy(&chunk);
        buffer.push_str(&text);

        // Process all complete NDJSON lines in the buffer
        while let Some(newline_pos) = buffer.find('\n') {
            let json_str = buffer[..newline_pos].to_string();
            buffer = buffer[newline_pos + 1..].to_string();

            if json_str.trim().is_empty() {
                continue;
            }

            if let Ok(response) = serde_json::from_str::<ChatResponse>(&json_str) {
                let content = &response.message.content;
                print!("{}", content);
                stdout().flush()?; // Print each part of the response as it arrives
                full_response.push_str(content);

                if response.done {
                    println!(); // Add a final newline
                    return Ok(full_response);
                }
            }
        }
    }
    
    // If the loop finishes but we haven't returned, something is off,
    // but we can return what we have.
    Ok(full_response)
}

This function does a few key things:

1. It constructs the ChatRequest with our system and user prompts.
2. It uses reqwest to send the request and get back a bytes_stream.
3. It loops over incoming data chunks, buffering them to ensure it only tries to parse complete JSON lines (ending in \n).
4. It deserializes each JSON line into our ChatResponse struct.
5. It prints the content to the console immediately for that "live typing" effect and also accumulates it into full_response.
6. When a response chunk has done: true, it returns the complete dialogue string.

4. Updating the Main Logic

Now, let's adjust our main function and construct_prompt to work with this new generate_dialogue function. We need construct_prompt to return the system and user parts separately.

Modify construct_prompt to return a tuple (String, String).

// Modify construct_prompt to return a tuple of (system_prompt, user_prompt)
fn construct_prompt(
    speaker: &CharacterContext,
    target_name: &str,
    interaction_name: &str,
) -> (String, String) {
    let persona_prompt = format!(
        "You are {}. Your personality traits are: \
        Honesty-Humility is {}. \
        Emotionality is {}. \
        Extraversion is {}. \
        Agreeableness is {}. \
        Conscientiousness is {}. \
        Openness to Experience is {}.",
        speaker.name,
        describe_personality_trait(speaker.personality.honesty_humility),
        // ... (rest of the personality traits)
        describe_personality_trait(speaker.personality.openness),
    );

    let situation_prompt = format!(
        "You are currently feeling {}. You are talking to {}, who is {}.",
        describe_mood(speaker.mood),
        target_name,
        describe_relationship(&speaker.relationship_with_target)
    );

    // This is now the 'system' part of the prompt
    let system_prompt = format!("{}\n{}", persona_prompt, situation_prompt);

    // This is the 'user' part of the prompt
    let user_prompt = format!(
        "You are performing the social interaction: '{}'. \
        Based on your personality and mood, generate a single, short line of dialogue (less than 20 words) that fits this interaction. \
        Your response must ONLY be the dialogue text itself. Do not include actions, descriptions in parentheses, or any surrounding text.",
        interaction_name
    );

    (system_prompt, user_prompt)
}

Note: Make sure to fill in all the describe_personality_trait calls that were abbreviated above.

Finally, update main to call our new function.

#[tokio::main]
async fn main() -> Result<()> {
    // --- Character Definitions (keep Alex and Bob from the previous lesson) ---
    let alex = CharacterContext { /* ... */ };
    let bob = CharacterContext { /* ... */ };

    // --- Scenario 1: Cheerful Alex greets his friend Bob ---
    let interaction = "Friendly Greeting";
    let (alex_system_prompt, alex_user_prompt) = construct_prompt(&alex, "Bob", interaction);
    
    println!("--- SCENARIO 1: Alex greets Bob ---");
    let alex_response = generate_dialogue(&alex_system_prompt, &alex_user_prompt).await?;
    println!("\n[Final Dialogue from Alex]: {}\n", alex_response);


    // --- Scenario 2: Grumpy Bob greets acquaintance Alex ---
    let (bob_system_prompt, bob_user_prompt) = construct_prompt(&bob, "Alex", interaction);

    println!("--- SCENARIO 2: Bob greets Alex ---");
    let bob_response = generate_dialogue(&bob_system_prompt, &bob_user_prompt).await?;
    println!("\n[Final Dialogue from Bob]: {}", bob_response);

    Ok(())
}

// Remember to include all the structs and helper functions:
// Message, ChatRequest, ChatResponse, CharacterContext, HexacoPersonality, RelationshipContext
// describe_personality_trait, describe_mood, describe_relationship

Make sure your local Ollama server is running, then execute the code with cargo run. You should now see the LLM "thinking" and then printing the generated dialogue in real-time, with each character's response tailored to their unique context.

Conclusion

Fantastic work! You have now built the complete round-trip for our dialogue system in Rust. We can define a character's state, construct a detailed prompt, send it to a local LLM, and get back a unique, in-character line of dialogue. This is the heart of the "deep systems" we aim to build for our game.

Key Takeaways:

• The Ollama chat API is ideal for role-based interactions, using system messages for context and user messages for instructions.
• Rust structs with serde make it straightforward to create and parse the JSON required for API communication.
• Handling streamed responses (NDJSON) requires a buffering strategy to process data chunks as they arrive.
• Asynchronous functions using tokio and reqwest are essential for non-blocking network I/O in a game engine backend.

Preview of the Next Lesson:
We've successfully generated dialogue text in our Rust backend. The next logical step is to get that text to appear in our game's UI. In the next lesson, "Integrate and display the generated dialogue text within the existing conversation UI," we'll focus on sending this data across the FFI boundary from Rust back to Unity and hooking it up to our user interface components.