Cross-Provider Chat Completion Requests and Field Normalization

Hello! In the previous lesson, you sent a fictional document question to Groq and inspected the assistant text, token usage, and Python-observed latency. Now you’ll send the same messages to Mistral through its Python SDK. Keeping the input fixed makes it easier to see which parts of the request and response are shared—and which parts your eventual multi-provider interface must hide.

Allow about 40 minutes. This is still a standalone generation test: there is no document ingestion or retrieval yet.

Check the Mistral request and response shape

Mistral’s chat-completion endpoint accepts a model ID and an ordered messages list, much like the Groq call you just made. The Python SDK method is different, however: this lesson uses client.chat.complete(...), whereas the Groq script used client.chat.completions.create(...).

Read the relevant portions of Mistral’s “Chat” API reference before coding. Concentrate on the ordinary, non-streamed completion; tools and streaming are outside today’s test.

The prior Groq script used max_completion_tokens=96; Mistral calls that request parameter max_tokens=96. Those names differ even though both set an output-token limit. We’ll leave sampling settings at each provider’s defaults, so this is a workflow comparison, not a controlled model-quality benchmark.

Make the equivalent request

Install Mistral’s SDK in the existing doc-qa project:

uv add mistralai

Put your Mistral credential in the ignored local .env, without committing it:

APP_PROVIDER=mistral
MISTRAL_API_KEY=your_actual_key_here

If config.py currently accepts only groq, extend its provider validation to accept mistral and expose the selected MISTRAL_API_KEY as settings.mistral_api_key. Mistral mode should not require a Groq key. Add an empty MISTRAL_API_KEY= entry to .env.example if you use that file to document required settings. As before, an environment variable already set in your shell may take precedence over .env.

Create mistral_smoke.py in the project root:

# mistral_smoke.py
from time import perf_counter

from mistralai.client import Mistral

from config import load_settings


def main() -> None:
    settings = load_settings()
    if settings.provider != "mistral":
        raise ValueError("Set APP_PROVIDER=mistral to run this Mistral test")

    model = "mistral-small-latest"
    messages = [
        {
            "role": "system",
            "content": "You are a concise assistant. Answer in one sentence.",
        },
        {
            "role": "user",
            "content": (
                "Test note: The sample library opens at 9 a.m. on weekdays.\n"
                "Question: When does it open on weekdays?"
            ),
        },
    ]

    # These fixed, fictional messages are safe to print for a smoke test.
    print("Messages sent:")
    for message in messages:
        print(f"  {message['role']}: {message['content']}")

    with Mistral(api_key=settings.mistral_api_key) as client:
        started = perf_counter()
        response = client.chat.complete(
            model=model,
            messages=messages,
            max_tokens=96,
        )
        elapsed = perf_counter() - started

    if not response.choices:
        raise RuntimeError("Mistral returned no completion choices")

    choice = response.choices[0]
    text = choice.message.content
    if not isinstance(text, str) or not text.strip():
        raise RuntimeError("Mistral did not return a nonempty text answer")

    print(f"\nReturned model: {response.model}")
    print(f"Returned role: {choice.message.role}")
    print(f"Generated text: {text}")
    print(f"Finish reason: {choice.finish_reason}")

    usage = response.usage
    if usage is None:
        print("Token usage was not returned.")
    else:
        print(f"Prompt tokens: {usage.prompt_tokens}")
        print(f"Completion tokens: {usage.completion_tokens}")
        print(f"Total tokens: {usage.total_tokens}")

    print(f"Measured SDK call time (s): {elapsed:.3f}")


if __name__ == "__main__":
    main()

Run it from the project root:

uv run --locked python mistral_smoke.py

The two messages are identical to the Groq test. The model ID and SDK call are deliberately provider-specific. If Mistral reports that the model is unavailable to your account, select a model you can access and change only model; an authentication or billing error instead calls for checking account access, not rewriting the prompt.

The text check is intentionally narrow. It verifies the plain-text answer this smoke test expects rather than silently treating a missing or differently structured response as a valid answer. Later, your common interface can make an explicit decision about other content types.

Decide what the application should—and should not—normalize

Mistral’s migration guide provides a concise view of why similar chat APIs do not imply identical Python calls. Its comparison is with the OpenAI-style client; your Groq script happens to use that style of chat method. This does not mean you should substitute one provider’s credentials or model ID for another’s.

Think of normalization as giving the rest of your document-QA application one dependable contract, rather than letting it know each SDK’s vocabulary:

The message roles and content can already have one application-level shape: an ordered list of system and user messages. The answer, model, finish reason, token counts, and measured latency are the response fields worth standardizing first. Request IDs and provider-specific timing details can remain optional metadata rather than becoming requirements every provider must satisfy.

Do not expect identical answers or identical token counts from the two runs. They use different models and may use different tokenizers. Also, Mistral’s usage counts are not a replacement for perf_counter() latency: one describes tokens; the other measures how long your Python process waited for the non-streamed call. Neither run measures time to the first visible token.

Finally, remove prompt printing before the application accepts actual questions or document text. It is useful here only because the test messages are fixed and fictional.

What you have established

You’ve now sent the same system and user messages through a second hosted provider, inspected Mistral’s completed response, and identified the boundary between shared application data and provider-specific SDK calls. Keep both smoke scripts: they give you small, independent checks when the common interface is introduced.

Next, you’ll make a comparable call to a local Ollama model. That third provider will test whether your proposed response contract works without a hosted API key.