Mapping Data/AI Workflows Across System Components

Hello! Last lesson turned the support-assistant pilot’s operating expectations into proposed requirements for latency, capacity, availability, security, privacy, and cost. You also have a functional boundary: agents receive sourced drafts only when approved policy supports them, agents control sending, and review-required replies need approval of their final text.

Now the question is where each responsibility lives. This lesson maps that workflow into source, processing, storage, model, service, and user-facing components. The aim is a logical architecture you could discuss with a customer before committing to particular cloud products. Plan on about 40 minutes, including the video, reading, and a careful look at the diagram.

Start with responsibilities, not product names

An architecture is more useful when its boxes describe work than when they merely display a collection of services. “Database,” for example, does not tell a reviewer whether it holds approved policy, a searchable copy of that policy, or draft history. Those stores have different owners, update rules, and access needs.

Watch ByteByteGo’s What is Data Pipeline? | Why Is It So Popular? for a compact vocabulary of sources, ingestion, processing, storage, and consumption. It describes data pipelines generally; we will add the model and application service needed for this assistant.

Microsoft Learn’s Big Data Architectures provides a complementary logical-component view. Its diagram contains alternatives, not a checklist of things every application must deploy. In particular, a small policy-document pilot does not need a streaming platform merely because streaming appears in a general architecture.

For the pilot, there are two kinds of input with different timing. Approved policy documents are maintained and prepared for later use. A support ticket arrives when an agent requests a draft. Keeping those paths distinct prevents an important mistake: treating every document that can be searched as policy that is approved for answering.

Map the support assistant’s six roles

Suppose an agent opens a ticket about a refund and asks the assistant for a reply. The architecture must know which policy applies, whether the agent may access the ticket, and what to display if the approved material does not support an answer. Here is a proposed logical map, not a claim that the customer already has these systems.

The distinction between a source and storage inside the solution matters. A search index is useful for finding passages quickly, but it should not silently become the authority on whether a policy is approved. If a policy is withdrawn, the system needs a defined way to stop using its indexed copy. Likewise, copying ticket text into an application database simply because a database exists would create a privacy and retention obligation.

“Model” and “service” also have different jobs, even if one managed product hosts both. The model produces or helps locate content. The service decides which inputs the model may receive, whether its output is safe to present as a draft under the agreed rules, and whether a later send action is allowed. A fluent model response cannot substitute for those application checks.

Trace the preparation path and the request path

The preparation path begins when a policy owner publishes or approves material. The proposed ingestion process reads eligible documents, records enough provenance to identify the source and version, and prepares a searchable copy. Its storage boundary is the approved policy material and its search representation—not an unrestricted collection of every document the organization happens to possess. How often this preparation runs depends on the freshness the customer needs; that cadence is still a question for the policy owner.

The request path begins in the agent’s interface. The backend verifies access to the ticket, obtains only the fields needed for drafting, searches policy material the request is permitted to use, and passes suitable evidence to the generation model. The service then returns a draft with references the agent can inspect. If it cannot find sufficient approved support, it returns a deliberate no-draft result instead. If search is unavailable, that is a system error, not evidence that no policy exists; the interface should not disguise one as the other.

Sending is another boundary. The agent may edit a draft, so approval of an earlier version cannot automatically authorize the final text. The send operation must check the agreed approval state of that final text. This is why the architecture includes an application service and, if approvals must persist across requests, a place to store their state. The model is not the enforcement point.

Now examine the supplied Google Cloud RAG architecture as an example of how these responsibilities can be deployed. It shows a data-ingestion subsystem on the left and a serving subsystem on the right. Read its service names as one implementation, not a shopping list for this pilot.

On the left, the raw-data bucket and processing job illustrate how documents can enter and be prepared for a managed data store. The message-and-trigger elements show one way to react to arrivals; they are not required if an agreed scheduled import is enough. On the right, a frontend and backend accept a user query, enterprise search prepares and searches for relevant information, and generation produces a response. The pictured custom agent is an additional option, not a necessary component of the support pilot.

Look particularly at what the image does not establish. It does not tell us which policy documents are approved, whether the querying agent can see the ticket, what counts as sufficient evidence, or who may send a reply. A cloud diagram can make data movement visible while leaving customer-specific rules undecided. An FDE’s job is to add those rules and their enforcement points, or mark them as unresolved—not to assume the managed components supply them automatically.

Make each handoff reviewable

A useful first architecture can be checked by following one policy change and one draft request across component boundaries. At each handoff, specify what crosses it, who controls it, and what happens when it fails.

Consider a policy update. The source must supply an approval status and an identifiable version. Processing must reject material outside the approved set and associate prepared text with its origin. Storage must make it possible to remove or supersede a searchable copy when the source changes. If the customer needs newly approved policy usable within a particular time, that freshness target becomes a requirement on this path; the previous lesson established how to make such targets measurable.

For a draft request, the interface supplies the agent’s action and the ticket reference. The service checks authorization before obtaining ticket details. Its model request should contain only necessary ticket information and permitted policy evidence, consistent with the pilot’s privacy constraints. The response needs enough source identification for the agent to inspect the support behind a draft. The interface must preserve the distinction among a supported draft, an intentional no-draft outcome, and a service failure.

This trace exposes where the earlier non-functional requirements apply:

• Latency and peak load concern the whole visible request, including service checks, search, model time, and rendering—not just model inference.
• Security and privacy concern both paths: who can import policy, who can retrieve it, which ticket fields leave the backend, and what is retained in logs or draft storage.
• Availability requires a manual support path when drafting fails, without weakening the rule that an unverified approval cannot authorize sending.
• Cost depends on which copies are stored, how often policy is prepared, how many searches and model calls each request makes, and what remains running between requests.

You do not yet need to specify an index algorithm, a message broker, or a cloud service. You do need to identify the handoffs that would let the team test the agreed behavior. For example, a citation requirement needs source identifiers to survive preparation and reach the interface. A final-text approval requirement needs a send operation that can verify the approved version. If either connection is absent from the map, the architecture cannot satisfy the requirement regardless of how capable its model is.

Conclusion

To map a data/AI workflow, begin with the user’s action and the authoritative data sources, then distinguish preparation from the live request. Assign each logical role a responsibility: sources provide authoritative inputs; processing prepares and selects them; storage preserves the needed forms and state; models retrieve or generate; the service enforces workflow rules; and the interface presents results and controlled actions. Products can be chosen after these responsibilities and handoffs are clear.

For the support-assistant pilot, the decisive boundaries are approval of policy before it is used as evidence, ticket authorization before its content is retrieved, and approval of final text before sending. Next lesson, you’ll turn architectural choices and unresolved questions—such as policy freshness and where approval state lives—into a concise decision record.