Tracing Security Requirements to Tests, Evidence, and CMVP Guidance

The trace is an assurance argument, not a list of documents

For CMVP purposes, a technical claim such as “the module protects sensitive security parameters” or “the module provides its required self-tests” cannot stop at an ISO clause. The laboratory must be able to assess the claim objectively. That requires a chain that connects the requirement to:

• what the vendor must supply;
• what the laboratory must verify, inspect, or test;
• which build, platform, and configuration the evidence represents;
• what a current, applicable IG changes or clarifies; and
• how the resulting conclusion is represented in the security policy and submission package.

The official CMVP overview states the underlying division of labor clearly: ISO/IEC 19790 specifies cryptographic-module requirements; ISO/IEC 24759 associates those requirements with vendor information and laboratory procedures; CMVP Special Publications manage CMVP-specific changes; and IG supplies interpretations that are often technology-specific.

A useful mental model is that the requirement is the claim, while evidence and test procedures establish whether the claim is substantiated for the particular module configuration.

The trace should preserve this separation. A test procedure is not itself the security requirement, and a document delivered to the laboratory is not automatically sufficient evidence merely because it has an appropriate title.

Establish the unit of tracing

A common early mistake is to create an enormous row labeled “self-tests,” “roles,” or “documentation.” These topics are too broad to close cleanly. Trace at the level of one assessable claim.

For example, these are workable trace units:

• the module performs a defined integrity test before cryptographic services are available;
• a specific role is limited to specified approved services;
• the module zeroizes a particular sensitive security parameter on a defined event;
• the documentation describes the module boundary and all logical interfaces;
• the module uses a specified approved algorithm implementation under stated operational conditions.

Each trace unit should answer a bounded decision:

For module version $V$, within boundary $B$, on configuration $C$, what must be true, what must be supplied, and how will the laboratory assess it?

This is particularly important for a multi-platform implementation. A statement about a shared cryptographic library may apply differently depending on the processor architecture, operating system, compile-time option, embodiment, or approved-mode configuration. If those facts are not frozen, the trace will look complete while silently joining evidence from incompatible configurations.

The minimum trace header

Before opening any standard or IG, create a header with the following facts:

Do not populate every column with “TBD” and proceed as if the trace were usable. An unresolved platform, boundary, or approved-mode fact is a trace risk that must remain visible.

Move from the normative clause to the assessment assertion

The technical requirement normally begins in ISO/IEC 19790, as used by FIPS 140-3. Record the exact clause or annex location, the applicable level, and a concise statement of the required security outcome.

Then locate the corresponding ISO/IEC 24759 assessment material. This normally gives you three linked elements:

• an assertion, identifying the claim to be assessed;
• vendor evidence, describing information the vendor must provide; and
• test evidence or a test procedure, describing what the tester must verify, inspect, or perform.

The identifiers matter. A trace containing only “ISO 24759 documentation section” is difficult to review and nearly impossible to defend when a laboratory asks which assertion a particular artifact supports.

The basic structure is:

where:

• $R$ is the normative requirement;
• $A$ is the assessable assertion;
• $VE$ is the required vendor evidence; and
• $TE$ is the laboratory’s test or verification procedure.

Evidence is not just “a document”

In consulting work, teams often translate every vendor-evidence requirement into “write a document.” That is too coarse. A vendor-evidence requirement may be satisfied through a controlled combination of:

• module specification;
• architecture or boundary diagram;
• interface table;
• source-code or configuration-management record;
• build manifest and version identifier;
• test plan, test result, or automated-test log;
• algorithm certificate reference;
• photographed physical sample;
• installation or administrator guidance;
• security policy table; or
• a laboratory demonstration of the implementation under a controlled configuration.

The required output is not merely an artifact name. It is an evidence locator: a specific document, repository tag, test-run identifier, configuration record, figure, or controlled package location that lets a reviewer retrieve the relevant proof.

Apply CMVP modifications before assigning evidence owners

The default ISO/IEC 24759 chain may not be the final CMVP chain. You must apply the relevant SP 800-140 treatment before telling a client what evidence to produce or what a laboratory is expected to test.

This is the point at which the previous lesson’s distinction becomes operational:

• SP 800-140 can modify the applicable vendor-evidence and test-evidence requirements associated with ISO/IEC 24759 requirements.
• SP 800-140A modifies documentation requirements tied to ISO/IEC 19790 Annex A and ISO/IEC 24759 paragraph 6.13.
• SP 800-140B modifies security-policy treatment associated with Annex B and ISO/IEC 24759 paragraph 6.14.
• SP 800-140C through SP 800-140F have explicit replacement roles within their identified annex domains.

For every trace, make the SP effect explicit:

The mechanism used in SP 800-140A is useful because it makes the relationship visible at the individual assertion level.

Worked trace: documentation requirement

The SP 800-140A excerpt gives a deliberately simple but important example. It is not a complete documentation package; it is one trace segment.

Two details deserve emphasis.

First, SP 800-140A does not say that a particular document title alone is enough. A file called “Security Policy” may still fail to contain required tables, accurate configuration information, or correct approved-mode instructions.

Second, the tester’s verification is not a substitute for vendor preparation. The laboratory does not reconstruct the intended architecture or invent missing configuration details. The vendor must provide a coherent and controlled evidence package capable of being evaluated.

Bring IG into the trace as a conditional constraint

IG is often inserted at the end of a matrix as a column filled with identifiers. That creates the appearance of completeness without an applicability decision. Treat IG instead as a conditional constraint on the trace.

For every potentially relevant IG item, record:

1. Identifier and revision — the exact item and current revision.
2. Issue addressed — the specific assertion, technology, or implementation behavior.
3. Applicability conditions — embodiment, boundary location, level, algorithm, interface, operational environment, validation state, or other stated conditions.
4. Effect — clarification, allowed approach, required evidence, required test treatment, documentation representation, restriction, or escalation condition.
5. Applicability result — applicable, not applicable, or unresolved.
6. Trace consequence — exact additional artifact, test observation, design condition, or security-policy statement required.

This makes the logic auditable:

For instance, if an applicable IG requires a particular technology-specific representation in vendor documentation, that result belongs in at least three places:

• the evidence cell, naming the artifact that contains the representation;
• the laboratory procedure cell, identifying what the laboratory will verify; and
• the security-policy impact cell, if the condition must be communicated to module operators.

Do not write “IG considered” as the conclusion. The conclusion must say what the IG does to this trace.

Three acceptable IG outcomes

“Not applicable” should never mean “not read.” State the mismatch that led to the conclusion.

Convert the chain into evidence and laboratory work

A complete trace must connect normative material to actual work products without assuming that the laboratory will infer the connection.

For each vendor-evidence item, assign:

• an artifact;
• a precise location or identifier;
• an evidence owner;
• a configuration or version binding;
• an internal quality-review status; and
• the corresponding laboratory verification method.

For each test-evidence item, identify the kind of activity it entails:

This distinction matters when a client asks, “What do we need to give the lab?” The answer should not be “all the documents.” It should identify what the laboratory needs to evaluate the specific assertion and how the artifacts collectively support the prescribed assessment procedure.

A seven-step tracing method

Use the following workflow for each atomic requirement claim.

1. Freeze the fact pattern

Record the module configuration, boundary, platform, claimed level, mode, and decision date. Reuse the source-currency discipline from the earlier lessons.

2. State the security outcome in plain technical language

Cite the normative clause, but also explain the outcome in one sentence. Avoid merely copying text without understanding the security property.

3. Locate the ISO/IEC 24759 assessment chain

Capture the assertion identifier, vendor-evidence requirement, and test-evidence requirement. Preserve their relationship; do not list them as disconnected citations.

4. Apply all in-scope CMVP modifications or replacements

Check the applicable SP 800-140 publication. Classify its effect as no change, addition, modification, deletion, or replacement.

5. Translate evidence requirements into controlled artifacts

For each vendor-evidence requirement, identify the actual artifact, its owner, its version binding, and its location in the evidence package.

6. Screen current IG and record its effect

Evaluate scope conditions explicitly. If applicable, update the artifact and test-treatment cells. If uncertain, create an open decision rather than a presumed interpretation.

7. Write a closure statement

The closure statement should be compact but substantive:

The specified module configuration satisfies the requirement because the identified implementation behavior is demonstrated by the listed controlled evidence, assessed under the applicable CMVP-modified laboratory procedure, and subject to the recorded IG conditions.

A trace is not closed merely because every cell contains text. It is closed when the evidence is available, internally consistent, configuration-bound, and sufficient for the laboratory procedure.

Reusable work product: the requirement-to-evidence trace record

Add the following template to your consulting toolkit. One row can be used for a small, simple claim; for a complex claim, use the header plus linked sub-rows rather than forcing everything into one oversized cell.

A practical review rule is to read the row from left to right and then from right to left.

• Left to right: Does the normative requirement lead to concrete evidence and laboratory activity?
• Right to left: Can every document, test result, and security-policy statement be justified by a requirement or applicable interpretation?

If an artifact cannot be connected backward to a requirement, it may be unnecessary or unmanaged. If a requirement cannot be connected forward to evidence and a test procedure, it is a validation risk.

Key takeaways

A high-quality CMVP trace links a security outcome to an assessable claim, controlled vendor evidence, laboratory verification, and any applicable IG conditions.

• Begin with an atomic, configuration-bound claim rather than a broad requirement area.
• Preserve the distinction between the normative requirement, the ISO/IEC 24759 assessment material, and CMVP modifications.
• Convert vendor-evidence requirements into specific, version-controlled evidence locators rather than generic document names.
• Treat IG as a conditional rule whose applicability and consequence must be stated explicitly.
• Close a trace only when the implementation, artifacts, laboratory treatment, and operator-facing documentation form one consistent assurance argument.

Next, the focus shifts from the trace itself to the people and institutions that make it work: assigning evidence-production, testing, review, decision, and escalation responsibilities among the vendor, accredited laboratory, NIST, and the Canadian Centre for Cyber Security.