Recommended resources
The freely available resources this lesson will direct you to.
Hello again. In the previous lesson, you built a risk register for an illustrative doorway-counting pilot. It identified what could go wrong, who would act, and which observations should trigger a pause. A release gate turns that preparation into a decision: is there enough evidence to expose the system to the next level of use and consequence?
This lesson defines two gates: prototype to pilot and pilot to production. You will learn what each decision must establish, what evidence to bring, and how to handle a result that is promising but incomplete. For a small team delivering edge analytics, the aim is not a lengthy approval ceremony. It is to prevent a successful demonstration from being mistaken for a safe pilot—or a successful pilot from being mistaken for a supportable product.
A gate is a decision, not a milestone on a calendar
A prototype shows that an idea can work under some conditions. A pilot tests whether a bounded solution works under agreed real-world conditions. Production means the organization is prepared to operate, support, and change that solution beyond the controlled trial. Each step changes the potential impact of a failure.
Watch What are Stage Gates or Gateways? by Mike Clayton of Online PM Courses. Its two questions provide a useful structure for a technical review.
What are Stage Gates or Gateways?
Watch for the distinction between completing a stage and authorizing the next investment.
Watch the gate concept for why a project pauses at a stage boundary. Then watch the two questions: has the current stage done what it needed to do, and is proceeding still justified? Apply both questions even when a customer is eager to begin.
For this course, a gate has five parts:
- Proposed scope: exactly what system, configuration, site, and use are being approved.
- Predeclared criteria: what must be true before the decision is made.
- Evidence: results and records that someone other than the builder can inspect.
- Decision authority: who may approve, block, or narrow the next stage.
- Recorded outcome: go, no-go, or a precisely bounded conditional go.
A date such as “pilot starts Monday” is a scheduling intention, not evidence. Nor is “the model looks accurate.” A useful gate statement sounds more like: “The specified application and model versions on the selected device passed the agreed preflight tests; media handling was checked against the approved data flow; the remaining limitation and its pilot response are documented.”
Keep three kinds of criteria distinct. Acceptance criteria judge whether the pilot delivered its promised outcome. Entry criteria judge whether starting the pilot is responsible. Production-readiness criteria judge whether the team can keep delivering that outcome and recover when something fails. Passing one does not automatically satisfy the others.
Gate 1: Is the prototype ready to become a bounded pilot?
A prototype-to-pilot gate does not require every production capability. It does require confidence that you can run a valid trial without exposing people, data, or the customer’s workflow to unexamined risks. The proposed pilot boundary matters: one approved camera and a supervised ten-shift evaluation is a different decision from unattended operation across many sites.
Begin with the question the pilot is intended to answer. In the doorway example, it might be whether alerts or counts reduce staff review work while meeting agreed recall, false-alert, and latency criteria. Before entry, confirm that the customer and team have fixed the observation conditions, baseline, reference-labeling method, minimum sample, exclusions, and interruption rules. Otherwise, even a smoothly running device may yield a result nobody can interpret.
Then ask whether the whole path works on representative hardware and inputs. A detector producing plausible boxes on saved clips is not enough. The selected edge device needs to receive the intended stream, generate the customer-visible event, and retain the timestamps and logs needed to assess it. A short preflight should include at least one expected failure, such as a camera disconnection. That test does not prove long-term reliability; it checks whether an obvious fault will be visible and handled during the trial.
Google’s SRE Workbook explains why teams use progressively more representative tests rather than treating a prototype as the final proof. Its discussion concerns data pipelines, so use the release logic—not its infrastructure choices—as the lesson for an edge device.
Google SRE - Improve and Optimize Data Processing Pipelines
Read the Google SRE Workbook’s treatment of development stages and end-to-end measurement. Focus on what each stage can reveal that the preceding one cannot.
In “Map Your Development Lifecycle,” read the release stages, from the “Prototyping” subsection through “Testing with a 1% dry run,” “Staging,” “Canarying,” “Performing a partial deployment,” and “Deploying to production.” Notice the changing realism of the inputs, the limited exposure during a canary, and the need to restore a known good state. Then, under “Pipeline Best Practices” → “Define and Measure Service Level Objectives” → “End-to-end measurement,” read the first paragraph. It begins by warning against relying only on per-stage measurements and ends by contrasting those measurements with the user experience. For the doorway pilot, consider why model inference time alone would not establish crossing-to-alert latency.
For a small team, an entry review can fit on one page. The following are minimum evidence categories, not universal numerical thresholds:
| Pilot-entry question | Evidence to inspect |
|---|---|
| Can the trial answer its stated question? | Approved scope, baseline, acceptance criteria, test conditions, labeling plan, and an achievable plan for collecting enough valid events. |
| Does the specified system work end to end? | A recorded run on the selected device and representative input showing capture, inference, event generation, and the customer-visible output. Include versions and configuration. |
| Can the team detect and respond to likely faults? | Results of preflight interruption and sustained-load tests; observable input gaps, process failures, and device-health signals; a named response owner. |
| Is live use permitted within the agreed boundary? | Confirmed installation and data-handling permissions, checked media destinations and retention behavior, and a way to stop the affected flow. |
| Is the pilot controllable? | Installation and recovery instructions, access to required logs, an escalation contact, and a risk register updated with test results. |
These checks connect directly to last lesson’s risks. R-01’s reconnection test and R-04’s sustained-load test are gate evidence. R-02’s traffic survey tells you whether the pilot can produce a valid recall measurement. R-03’s media-path review is a hard blocker before live media: promising detection results cannot compensate for an unapproved data flow.
The threshold for a preflight test should be set before running it. For example, the earlier risk register used an illustrative trigger of a failed automatic reconnect or an input gap over two seconds during a one-hour disturbance test. If that trigger fires, the result is not “mostly passed.” It is a reason to fix, retest, and reconsider pilot entry. The exact threshold belongs to the agreed stream and pilot conditions.
Gate 2: Did the pilot justify production—and can the team operate it?
A pilot-to-production gate asks two independent questions. Did the system achieve the agreed pilot outcome? And can you deliver and sustain that outcome at the proposed production scope?
First, close the pilot evaluation without moving the goalposts. Compare results with the predeclared baseline and targets, using the agreed observation window and exclusions. Report sample sizes and missing periods alongside metrics. If the doorway pilot required at least 200 independently labelled crossings but produced only 180, recall acceptance is inconclusive under that rule—even if the observed recall looks excellent. Inconclusive is not the same as failure, but it is not a pass. The next decision might be an agreed extension under the original measurement method, rather than a retrospective reduction of the minimum.
Assess the customer’s workflow as well as model output. If the intended benefit was less staff review time, count how much review the delivered system actually required. Crossing-to-alert latency should include capture, processing, and delivery, not just inference. Record false alerts and missed events in the conditions where the system will be used. If some shifts were affected by input gaps, apply the interruption rule rather than quietly removing inconvenient periods.
Second, examine operational evidence. A production service needs more than a good pilot average. Someone must know which version is deployed, detect a stopped camera or full storage device, restore operation, and distinguish a model-quality problem from an input or device fault. For the proposed production scope, ask for:
- Repeatable deployment: a recorded application, model, device-software, and configuration combination; an installation procedure that another team member has followed.
- Observed reliability: pilot logs and incidents, including uptime or valid observation time, input gaps, resource use, and how long detection and recovery took.
- Detectability and response: actionable alerts or checks for critical failures, named responders, and instructions for the faults encountered in the pilot.
- Change and recovery control: a known good version and a demonstrated way to restore it if an update or configuration change fails.
- Data and access controls: reviewed access, destinations, retention, and deletion behavior for the production use that is actually proposed.
- Capacity and support: evidence that the device and team can handle the proposed number of streams, sites, and support hours—not an extrapolation from one short demonstration.
A pilot may reveal that the product works well only at its pilot boundary. Approval for one supported device and stream must not be recorded as approval for an unattended fleet. If production will add new hardware, operating conditions, or scale, require additional representative testing or a limited initial rollout. The SRE reading calls this kind of constrained exposure a partial deployment or canary; the practical principle is to discover problems while the affected scope is still small.
You do not need to build an elaborate operations platform before serving a first customer. You do need controls proportionate to the promise you are making. A manual recovery procedure can be acceptable for a narrowly supported deployment if its owner, response time, and limitations are explicit. “An engineer will figure it out” is not a recovery procedure.
Make the decision reviewable—and allow it to say “not yet”
A gate meeting should inspect a small evidence packet, not rely on a live demonstration. For each criterion, record its owner, the test condition, the result, and where the supporting artifact can be found. Artifacts might include a signed-off evaluation report, timestamped event export, device telemetry, configuration record, test log, or installation checklist. Record failures as well as passes.
The decision record can be short:
Decision: No-go for production of the proposed doorway service.
Scope reviewed: Specified application and model versions, selected edge device, one pilot stream and site.
Evidence: Agreed outcome metrics met on valid shifts; device recovered from observed camera interruptions. Installation by a second team member was not demonstrated, and no recovery procedure was tested after a failed update.
Required before re-review: Complete those two checks, attach results, and review the updated risk register.
Decision owner and date: Recorded by the person authorized to approve production scope.
This is not a verdict on whether the pilot was worthwhile. It says the pilot established value while exposing an operational gap.
Use conditional go sparingly. It must state the permitted scope, the unmet criterion, the compensating control, its owner and deadline, and the condition that ends permission. For instance, a supervised pilot might proceed with a nonessential dashboard deferred if the team can inspect the same required evidence another agreed way. An unapproved media path, absent authority to use live inputs, or inability to observe critical failures is not a sensible candidate for a conditional go.
A simple decision rule keeps the meeting honest: a missing measurement is not a passing measurement; a risk accepted for a bounded pilot is not automatically accepted for production; and a change in scope requires revisiting the evidence. The person accountable for the gate should be able to pause the release, even when the schedule is uncomfortable.
Key takeaways
The prototype-to-pilot gate establishes that a bounded, permitted, measurable trial can run on the intended system, with visible faults and owned responses. The pilot-to-production gate establishes both that the agreed outcome was demonstrated and that the organization can repeatedly deploy, observe, support, and recover the service at its proposed scope. Define criteria before collecting results, preserve the evidence behind the decision, and distinguish pass, fail, and inconclusive.
This completes the module’s decision-making framework. Next, you will build the Linux and software-engineering skills needed to produce much of this evidence yourself: inspecting devices and processes, packaging changes, and diagnosing failures rather than relying on a demonstration alone.