What to hand an auditor

Most vendor material about audit evidence describes a document. That is the wrong unit. An auditor is not collecting documents, they are testing assertions, and an artefact is only useful to the extent it supports one. This page sets out what the assertions usually are, which of our outputs bear on them, and where the line falls between what we can say and what only an independent practitioner can.

#What an auditor is actually testing

Whether the engagement is a SOC 2 examination, an ISO 27001 certification audit, an internal audit review or a supervisory inspection, the underlying tests are similar. A control owner asserts something. The reviewer tries to determine whether the assertion holds, using evidence they did not create.

The order below is roughly the order in which questions get harder to answer, and the last two are where cryptographic evidence usually falls apart.

1. 01Existence. Does the thing you say you have actually exist? Show me the inventory.
2. 02Accuracy. Are the entries right? The reviewer picks a sample and traces each one back to the system it describes.
3. 03Completeness. Is anything missing? This is the difficult one, because it asks you to characterise what you did not find.
4. 04Timing. Was the control in place across the whole period under review, or only on the day you were asked?
5. 05Authorisation. Who approved the exceptions, and on what basis? An accepted risk with a named owner is evidence. An unexplained gap is a finding.
6. 06Consistency. Does this artefact agree with the others? A cryptographic inventory that contradicts the asset register creates two findings rather than none.

#Inventory, attestation, opinion

These three words get used interchangeably in sales material and they are not interchangeable. The distinction is who is asserting what, under which professional standard, and therefore who may rely on it.

The practical consequence: if your customer's procurement team asks for a SOC 2 report and you send a Posture Attestation, you have not answered them. If your internal auditor asks for evidence that your cryptographic inventory is complete, a Posture Attestation is relevant and an inventory export is more relevant still.

#The artefacts we produce

Each row below names what the artefact evidences and what it does not. The third column is the one to read before you put something in an evidence pack.

#Completeness is the hard question

Every experienced reviewer asks the same thing about an inventory: how do you know this is all of it? A vendor answer of yes is worthless, because completeness of a discovery process is not something a discovery process can establish about itself.

The credible answer has three parts, and it is worth rehearsing before the meeting. State the scope that was agreed and what fell outside it. State the measured error rate of the classification rather than claiming there is none. State what a second, independent method found when run over the same estate.

• Scope. Named systems, named repositories, named network ranges, named exclusions, with the exclusions attributed to whoever decided them.
• Measured error. On our labelled evaluation set the classifier achieved 95.2% accuracy and 0.94 macro-F1. Those are the numbers we quote, including when they are inconvenient, and the methodology is in Accuracy, false positives, and how we report error.
• Corroboration. In our pilot estate, discovery returned 42 cryptographic assets, of which 36 were classified vulnerable, for an aggregate risk score of 75 / 100. The same estate took 22 hours of manual review against 14 minutes of AutoPQC time. The manual pass is what makes the automated result checkable.
• Exceptions. Every asset the tool could not classify, listed as unresolved rather than absent. A short unresolved list is a sign of a working process. An empty one is a sign of a hidden assumption.

#Handing it over

The sequence matters more than people expect. Most disputes in a review come from an artefact whose scope was never agreed, or a snapshot that moved under the reviewer while they were sampling it.

1. 1

   Agree the scope in writing first

   Before discovery runs, get the reviewer to confirm which systems, environments and repositories are in the population. A scope agreed afterwards is a scope the reviewer will question.

2. 2

   Run discovery and work the exceptions

   Resolve or explicitly accept every unclassified asset, with a named owner per acceptance. Do this before the reviewer sees the list, not during their sampling.

3. 3

   Freeze a snapshot

   Fix a dated, version-stamped snapshot and hand over that, not a live view. A reviewer sampling a changing dataset cannot conclude anything, and will say so.

4. 4

   Export the population and the method together

   The inventory export and the method revision it was produced under travel as one package. An inventory without a method is a spreadsheet of assertions.

5. 5

   Walk the reviewer through one finding end to end

   Pick a single asset and trace it from the detector evidence to the score to the remediation entry. This answers more questions than an hour of overview. Why a specific asset was flagged is written for exactly this conversation.

If a reviewer wants to see the process rather than the output, we will join the call. We do not charge for that and we do not need to be the ones talking.

#Verifying what someone hands you

The reverse case comes up more often than the forward one: a counterparty sends you a Posture Attestation and you have to decide what it is worth. Every attestation we issue carries an identifier that can be checked at Attestation Lookup without an account, without contacting us, and without visibility into the estate it describes. The lookup confirms that the identifier was issued by us, the method version it was scored under, and its current standing. It does not disclose findings, asset names or any content of the underlying report. [PLACEHOLDER: the signing key and algorithm published for independent verification of attestation payloads]

A verified identifier tells you the attestation is genuine. It does not tell you the estate is secure, and it does not tell you the scope was adequate. Read the scope statement. An attestation over three systems in a two-hundred-system estate is genuine and close to meaningless, and the only way to tell the difference is to look.

If the question is about us rather than about a customer estate, the Assurance Center holds our own control posture, our subprocessors, our data-handling position and an explicit list of the certifications we do and do not currently hold. We would rather you read that list than assume it is longer than it is. Vulnerability reporting and our disclosure policy sit at Security & Disclosure.