EQTY LabDocs

Agent Assessments

Assess an agent against a Governance Program version, submit its evidence for review, and have reviewers judge each control.

View as Markdown

About agent assessments

An assessment is an agent’s case for certification under one version of a Governance Program. The agent’s owner starts it by pinning the configuration the agent runs, and optionally its operating profile, then declares how the agent meets each control and submits the assessment. The program’s reviewers judge each control of the submission, and once every control passes, the agent is certified.

An assessment is of one agent, never of a project. Certification is self-service: any member who can see an active agent in Guardian can start an assessment of it, which normally means its owners. Only whoever started an assessment changes and submits it, and only the program’s assigned reviewers review it.

Declarations, submissions and reviews are signed and registered with the Integrity Service as you make them, so they can’t be edited or deleted afterward. An assessment can’t be removed once started.

Key actions associated with assessments include:

Starting an assessment

The agent must be active. To pin its configuration, it must also have sent requests through the Agent Gateway in the last 90 days, since the gateway observes the configurations it runs.

To start an assessment:

  1. Select Agents in the sidebar, then the agent
  2. On the agent’s Assessments page, select Start Assessment at the top. On its Overview, select Actions, then Start Assessment.
  3. Choose the Program and, if it has more than one version, the Version. The assessment stays judged against that version, whatever versions are published later.
  4. Choose the Configuration to pin. See Pinning a configuration.
  5. Choose the Operating Profile to pin, or None. See Pinning an operating profile.
  6. Select Start Assessment

You can also start one from the Assessments page: select Start Assessment in its header and choose the Agent first, from the active ones, then the program. The page offers this only with the Agent Gateway, which lists the agents.

If the agent already has an open assessment against the program, the dialog says so, with Open to go to it (the newest, if there are several). Continue that one rather than starting another: an assessment can’t be removed once started.

Starting pins the configuration and operating profile in the same step. If either can’t be pinned, nothing is started: the dialog says why and keeps your choices, so you can change them and select Start Assessment again.

The assessment opens in its workbench, ready for declarations. The explorer on the left lists the program’s controls. Selecting one shows it in the pane on the right, open on Control Details: what the control asks and the evidence it’s judged on. Its record of what was declared and reviewed is one click away, under Control Timeline. The place to declare against it is below both.

Pinning a configuration

Reviewers judge the agent by a configuration it ran, which you pin when you start the assessment. A configuration is identified by its configuration CID, a fingerprint of the configuration fields the agent’s requests captured: the same settings always give the same CID, and any change to them gives a new one. See Agent Configuration for what is captured.

The Configuration field lists each configuration the agent ran in the last 90 days once, by its configuration CID, however many requests ran it, newest first, and starts on the newest. Beneath it, the dialog says what the chosen configuration runs. A configuration that didn’t capture a field the program’s scope requires can’t be pinned, and the dialog names what it’s missing. If none of the agent’s keys ran one in that time, the dialog asks you to run the agent through the gateway again first.

The certification the assessment earns names the pinned configuration, and only that configuration: if the agent later runs with changed settings, the certification doesn’t cover them. Pin the configuration the agent will run once certified. The pin can’t be changed once the assessment starts, except when submitting fails because the gateway can no longer serve it: the workbench then asks you to pin the configuration again, on its Configuration tab.

An agent whose requests predate the gateway recording configurations lists its observed states instead.

Pinning an operating profile

An operating profile is the owner’s declaration of what an agent is for: its purpose and accountable team, the context it runs in, the systems it reaches and its access to them, and what it must never do. You design and version it on the agent’s Operating Profile page; see Operating Profiles.

Pinning a profile is optional. When you pin one, reviewers judge the agent against it, the certification names it, and the gateway holds the agent to it. Without one, a control judged against profile sections needs a declaration or a not-applicable claim.

The Operating Profile field starts on the agent’s current profile version. Choose another saved version, or None to start without one. Like the configuration, the pin can’t be changed once the assessment starts, unless submitting fails because the agent no longer holds the pinned profile.

Declaring against controls

A declaration is a signed statement, any number of files up to 20, or both, describing how the agent meets one or more controls. The program’s Expected Evidence for a control says what the owner is expected to declare.

To add a declaration:

  1. Select the control in the explorer
  2. Under Declare, the declaration form is already open for a control nothing covers yet. For a control already covered, select Add Declaration to open it
  3. Enter a statement describing how the agent meets the control, and attach files to support it
  4. To cite the same declaration for other controls, open Also covers other controls and choose them
  5. Select Sign and Add

A control judged only against configuration fields, with no expected evidence, needs no declaration: the pinned configuration addresses it, once it captured those fields. The same holds for a control judged against profile sections, while a profile is pinned.

To take a declaration off the draft, select Withdraw from Draft on it. It comes off every control it covers. Submitting locks in the draft’s declarations and claims: you can withdraw what you’ve added since the last submission, and a submitted one only once a reviewer has found a control it covers Not Satisfied in the newest submission, so you can replace it.

Guardian shows an assessment, its declarations, and their files only to the assessment’s readers: Organization Owners, whoever started it, and the program’s reviewers.

Claiming a control is not applicable

Where a program permits it for a control, you can claim the control doesn’t apply to the agent instead of declaring against it. The program’s Not Applicable When says when that holds.

  1. Select the control in the explorer
  2. Under Not applicable, select Claim Not Applicable
  3. Under Why doesn’t it apply?, explain why, citing the evidence. A reviewer judges whether the claim holds.
  4. Select Claim Not Applicable to confirm

Select Withdraw Claim to take a claim back. As with declarations, a claim a submission froze is locked in until a reviewer finds the control Not Satisfied in the newest submission.

Submitting for review

When the configuration is pinned and every control is addressed by a declaration, a not-applicable claim, or what is pinned, select Submit for Review. The workbench returns you to the list you opened it from.

Submitting freezes the draft into a numbered submission: the pinned configuration and operating profile, the declarations, the not-applicable claims, and the files the configuration names, so reviewers who can’t see the agent can still read them. If a profile is pinned, the submission also records how the profile compares with what the agent was seen doing, for reviewers to weigh.

Submitting is blocked while no configuration is pinned, while a control has no declaration and no not-applicable claim, or when nothing has changed since the last submission. Hover over the disabled Submit for Review to see which.

A submission can’t be changed. To change what you submitted, keep working on the draft and submit again. The new submission supersedes the earlier one, which can then no longer be reviewed or certified, and reviewers start again on the new one. An assessment’s status always follows its newest submission:

StatusMeaning
DraftNever submitted
In ReviewThe newest submission has a control no reviewer has reviewed yet
PassedEvery control’s newest review of the newest submission is Satisfied or Not Applicable
FailedEvery control of the newest submission is reviewed, and at least one isn’t satisfied

You keep working on the draft after you submit. In the explorer, each control you haven’t changed since the newest submission shows that submission’s verdict, or that it’s awaiting review. Reviewers and everyone else open on the newest submission, its status beside the title, or its certification once one is issued. A certification’s link opens the submission that earned it, for whoever started the assessment too, with a banner back to the draft or the newest submission. Only the newest submission can be reviewed. Earlier submissions stay on record in each control’s Control Timeline, with every declaration, verdict and certification made against them. Every pin of the configuration and operating profile is on record there too, with who pinned it, when, and what it changed from the pin before.

Reviewing a submission

Reviewers are assigned to a program by an Organization Owner (see Assigning reviewers). A reviewer can’t review an assessment they started, or a submission they made.

Select Agents in the sidebar, then Assessments. The page has two tabs:

  • Assessments lists every assessment you can read: those you started and those of the programs you review, or every assessment for Organization Owners. Filter it by Program, Status or Started By.
  • To Review lists the submissions waiting on your review, and counts them on the tab. Filter it by Program or Started By.

The certifications those assessments earned are under Agents, then Certifications.

To review a control:

  1. Open the assessment from To Review, and select a control in the explorer
  2. Read the control’s evidence: the configuration fields and profile sections it’s judged against, and any not-applicable claim the submission froze. Control Details says when the program permits a control not to apply. Its declarations are in the Control Timeline, beside the verdict.
  3. Under Your verdict, choose Satisfied, Not Satisfied, or, where the program permits it, Not Applicable
  4. Enter a statement. It’s required for Not Satisfied: say what’s missing and what the owner should change.
  5. Select Sign and Record

A control can be reviewed again, and its newest review counts. Each review is signed and recorded permanently.

When your verdict is the last one the submission needs to pass, the button reads Sign and Certify, and confirming issues the agent’s certification.

Issuing the certification

The review that leaves the newest submission passed asks for its certification, with the reviewer as the person who authorized it. The Guardian Platform signs the certification with the platform key on the reviewer’s behalf, records it, and attaches it to the Agent Gateway. The certification expires one certification lifetime after that review, or never when the program version sets no lifetime, and is valid from an hour before it is issued, to allow for clock skew, but never before that review. A submission whose term would already have ended can’t be certified.

Signing with the platform key needs an Audit Owner, Organization Owner, or Project Owner role. A reviewer working through an API key signs only when the key’s scopes cover every permission of such a role. When the reviewer doesn’t hold one, or issuing fails for another reason, their review still stands but the agent isn’t certified. An Organization Owner, or a reviewer whose review the certification rests on and who can sign, then selects Issue Certification in the workbench’s top bar. The member who started the assessment never issues its certification, even as an Organization Owner, so another Organization Owner issues it for them. Asking again for a submission that was certified returns the certification it already has.

Only the newest submission of a passed assessment can be certified. Once a submission is certified, it takes no more reviews; to certify the agent again, for example under a new configuration, submit again or start a new assessment.