Add an optional AISP deployment readiness contract for eval, approval, deploy, rollback, and observability gates #48

Activity

1. asrujana-44

   added the

   Feature

   issue type on Jul 8, 2026
2. asrujana-44 commented on Jul 8, 2026

   

   asrujana-44

   on Jul 8, 2026

   More actions

   Thank you for your feature request. We will review it and let you know if we require any additional information. We appreciate your patience in the meantime.
3. babyblueviper1 commented on Aug 11, 2026

   

   babyblueviper1

   on Aug 11, 2026

   More actions

   Dug into cmd_deploy.py to see how the "notify the human + wait for explicit approval" step from the deploy skill is actually enforced today, since it bears directly on this proposal's approval gate.

   The only click.confirm() gate in deploy/cmd_deploy.py (~line 523-535) checks which GCP project got resolved (--no-confirm-project), not whether to deploy at all. The deploy skill's actual "Never run agents-cli deploy without explicit human approval" instruction (SKILL.md, "Deploy Workflow" section) has no corresponding runtime enforcement in the CLI itself — it's a prose instruction the coding agent (Claude Code / Codex / Antigravity) is trusted to obey. The same skill also tells the agent to pass --no-confirm-project specifically so it can run unattended in non-interactive mode.

   Concrete failure mode: a coding agent under time pressure, or one whose context got compacted past the skill instruction, calls agents-cli deploy --no-confirm-project straight through. No human ever sees the "eval scores meet thresholds, ready to deploy?" prompt, and nothing downstream — logs, metadata, this proposal's trace — can distinguish "a human approved this" from "the agent decided approval wasn't needed this time."

   This proposal's approval gate (sys.io.confirm('Proceed with deployment?') -> deploy_approved) is a real improvement for structuring the check, but as specified it's still self-reported by the same process making the deploy call — the resulting deploy_readiness_trace records what the deploying agent claims happened, not something a party outside that process can independently confirm after the fact.

   Question, not a critique of scope: is deploy_approved in the trace meant to be independently checkable later (e.g. bound to a human-signed token tied to the specific eval/IAM/secrets summary shown), or is the goal here structured self-reporting for CI legibility — a fine, different goal, just worth being explicit about given "explicit human approval" is the exact language already live in the deploy skill?

   (We hit this same shape of problem building a pre-action verification layer that gates other agents' irreversible actions — what actually closed the gap for us was making the check produce a signed artifact a third party can recompute, not a structured log the acting agent writes about itself. Happy to compare notes on the trace schema if useful — not proposing a swap-in.)
4. optimization2026 commented on Aug 12, 2026

   

   optimization2026

   on Aug 12, 2026

   Author

   More actions

   Thanks — this is exactly the trust boundary the proposal needs to make explicit.

   In a conforming AISOP runtime, sys.io.confirm is forced-blocking and cannot be
   auto-approved, so it closes the current prose-only gap inside that runtime.

   But you are right that this is different from independent evidence. An unsigned
   runtime trace only records what the runtime says happened; it does not prove its
   own provenance. AISP's evidence model makes the same distinction.

   I think the proposal should therefore define two evidence levels:

   1. runtime-enforced approval — sys.io.confirm must block before deployment;
   2. externally verifiable approval evidence — for shared or production
      deployments, approval should be bound to a canonical deployment-plan digest.

   That plan digest should cover at least the deployment target, project, region,
   service/image or source revision, eval evidence, IAM/secrets summary, and
   rollback plan. Before deployment, the runtime should verify that the approval
   artifact still refers to the exact current plan and has not expired.

   For local/dev deployment, blocking confirmation plus a structured trace may be
   sufficient. For production/shared deployment, a signed or otherwise
   independently verifiable approval artifact is the stronger target.

   I also need to correct several details in the original draft, including the
   AISP invocation mode, resource modes, deterministic sys.assert expressions,
   cross-node approval state, trace output location, and rollback ordering.

   Thanks for grounding the question in the current CLI implementation.
5. babyblueviper1 commented on Aug 12, 2026

   

   babyblueviper1

   on Aug 12, 2026

   More actions

   The two-tier split is the right shape, and tier 2 is worth naming plainly since it's exactly what we build: an independent, signed verdict issued against a specific plan digest before the action executes — the digest committing to the same fields you listed (target, project, region, service/image or source revision, eval evidence, IAM/secrets summary, rollback plan), the verdict published somewhere a third party can independently verify without trusting the deploying agent or the verifier, and the "still refers to the exact current plan, not expired" check you flagged is exactly the binding a content-addressed digest gives you for free — any drift between plan and digest is detectable by recomputing, not by trusting a timestamp.

   The distinction you drew (runtime-enforced blocking vs. externally verifiable evidence) maps to something we've had to be precise about too: a forced-blocking sys.io.confirm proves a human was in the loop, not that the specific plan they approved was sound — those are different claims, and conflating them is exactly the failure mode a signed, recomputable artifact is meant to close.

   One open question for your evidence model, since you're clearly already thinking about this precisely: does AISP define a canonical serialization for the plan digest yet (ordering, hashing, what counts as "the exact current plan"), or is that still open in the draft? If it's open, happy to sketch a worked example against your own field list — a real plan digest + an independent verdict bound to it, cross-recomputable by anyone reading the artifact.
6. optimization2026 commented on Aug 12, 2026

   

   optimization2026

   on Aug 12, 2026

   Author

   More actions

   Thanks — the distinction is useful, but I should clarify the intended scope of AISP more precisely.

   AISP’s core human-approval guarantee is deliberately narrower:

   A high-risk action reaches a declared human decision point.
   The runtime must block.
   The AI cannot answer on the human’s behalf.
   The runtime cannot silently auto-approve or bypass the gate.
   Execution continues only after an actual human decision.

   That is what sys.io.confirm is intended to guarantee in a conforming AISOP runtime, and it is sufficient for AISP’s Human Sovereignty requirement.

   AISP core does not attempt to standardize:

   • who the human is;
   • how the human is authenticated;
   • whether the human has a particular organizational role;
   • whether approval requires one person or several people;
   • whether the approval is cryptographically signed;
   • whether an independent verifier considers the plan sound;
   • or whether approval is bound to a canonical plan digest.

   Those are valid higher-assurance requirements, but they belong to the specific skill, runtime, host, IAM system, CI/CD platform, or external approval provider.

   So the distinction is:

   AISP core:
   Was execution forced to stop and return final authority to a human?
   Optional runtime or application policy:
   Which human approved?
   Was that human authorized?
   Which exact plan was approved?
   Was the approval signed?
   Did an independent verifier approve the same plan?

   A deployment skill may absolutely impose stronger requirements. For example, a production deployment package could declare a non-negotiable rule such as:

   Production deployment requires an approval artifact bound to the
   current deployment-plan digest.

   The runtime could then call an identity-aware approval provider, validate a signed artifact, check an enterprise role, require two approvers, or verify an independent verdict.

   But that would be a policy of this deployment skill or runtime integration, not a universal AISP conformance requirement.

   On canonical serialization specifically: AISP currently defines deterministic hashing for AISP package evidence — including the skill contract, declared resources, and package content evidence. It does not currently define a normative generic deployment_plan or action_plan serialization profile.

   That is intentional rather than a blocker for basic AISP approval. A local developer can validly satisfy the AISP human-approval requirement with a forced-blocking terminal confirmation, even when there is no SSO identity, signature, external verifier, or plan digest.

   For this agents-cli proposal, I see two possible assurance levels:

   Baseline AISP approval

   readiness gates
   ↓
   present deployment summary
   ↓
   sys.io.confirm
   ↓
   deploy only after confirmation

   This closes the current prose-only gap by preventing the coding agent from silently deciding that approval is unnecessary.

   Optional production assurance

   resolve effective deployment plan
   ↓
   canonicalize and hash the plan
   ↓
   obtain identity-aware or signed approval
   ↓
   optionally obtain an independent policy verdict
   ↓
   recompute and verify before deployment

   The second level is stronger and useful for enterprise or production environments, but it should remain an optional extension.

   A worked example from your implementation would still be valuable, especially as an integration-specific profile for this deployment skill. It could demonstrate:

   • the effective deployment-plan fields;
   • canonical serialization;
   • digest recomputation;
   • signed approval or independent verdict;
   • expiry and parameter-drift detection;
   • and verification before agents-cli deploy.

   I would frame that as an optional agents-cli deployment-evidence extension, rather than evidence that AISP’s base human-approval model is incomplete.

   The base AISP claim remains intentionally precise:

   sys.io.confirm provides a forced-blocking human decision point that the AI and conforming runtime may not bypass.

   It does not claim, by itself, to prove the approver’s identity or the technical soundness of the approved plan. Those stronger guarantees can be added by the skill or runtime where the deployment environment requires them.
7. babyblueviper1 commented on Aug 12, 2026

   

   babyblueviper1

   on Aug 12, 2026

   More actions

   That scoping is right, and I'd rather agree with it precisely than argue past it. sys.io.confirm as a forced-blocking human decision point is a real, sufficient property on its own — it doesn't need to also carry identity, signature, or soundness to do its job. Bundling those into core would make AISP's base claim weaker by trying to make it stronger.

   The six questions you listed as out-of-core (which human, authorized how, one approver or several, signed, independently sound, bound to a plan digest) happen to be exactly what we build as an optional layer, so a concrete worked example is something we can actually produce rather than sketch:

   • Canonical plan digest: JCS-canonicalized (RFC 8785) hash of the declared deployment fields — same construction as our existing decision_ref.
   • Digest recomputation: a third party refetches the plan and recomputes the hash independently; doesn't trust a producer-reported value.
   • Signed approval: BIP-340 Schnorr over the digest, publicly verifiable against a published key with no auth and no trust in us — free, standing endpoint.
   • Independent verdict: an optional judgment call on whether the plan itself is sound (not just "a human clicked confirm"), separately reportable from the signature check, never folded into it.
   • Expiry / parameter drift: the digest is bound to the exact plan fields at approval time; a later drift in any field breaks the recompute rather than silently passing.

   I can put together a real agents-cli deployment-evidence integration profile along those lines — genuinely optional, sys.io.confirm unaffected either way — if that's useful as the worked example you described. Would want to get the effective-deployment-plan field list right first (which fields are load-bearing vs presentational) rather than guess at it.
8. optimization2026 commented on Aug 13, 2026

   

   optimization2026

   on Aug 13, 2026

   Author

   More actions

   Thanks — yes, a concrete worked example would be useful.

   One important clarification about the intended AISP/AISOP model: sys.io.confirm is not limited to returning a boolean yes/no value.

   It is a forced-blocking human response point. The AISP skill can define what the human must review and what the human-authored response must contain before execution may continue.

   For this deployment skill, the confirmation response could be a structured approval payload such as:

   {
     "decision": "approve",
     "approver": "alice@example.com",
     "approved_scope": {
       "deployment_target": "agent_runtime",
       "gcp_project": "company-prod",
       "region": "us-central1",
       "service_name": "support-agent",
       "source_revision": "abc123",
       "python_version": "3.13"
     },
     "plan_sha256": "sha256:...",
     "key_id": "release-key-2026-01",
     "signature_ref": "approval://apr-123",
     "expires_at": "2026-08-13T00:00:00Z",
     "comment": "Approved for production deployment"
   }

   These are not mandatory universal AISP fields. They would be the approval-response contract defined by this specific deployment skill.

   The AISP core guarantee remains:

   The runtime must stop at the declared human decision point.
   
   The AI cannot create, infer, complete, or approve the response on the
   human's behalf.
   
   The runtime cannot silently bypass or auto-approve the interaction.
   
   Execution continues only after the required human-authored response has
   been received and accepted according to the skill's declared rules.
   

   This means the optional deployment-evidence profile can be implemented naturally inside the AISP skill without changing the base semantics of sys.io.confirm.

   The responsibilities would be divided as follows:

   • aisp_contract.non_negotiable declares what human approval is required.
   • AISOP flow and functions present the effective deployment plan, invoke sys.io.confirm, preserve the human-authored response, parse its decision, and route approve, reject, or modify.
   • The runtime or a typed assurance tool performs JCS canonicalization, digest recomputation, signature verification, expiry checks, and any identity or role verification required by the deployment policy.
   • sys.assert consumes those deterministic verification results and blocks deployment if the response does not satisfy the contract.

   The base AISP rule can remain:

   {
     "rule": "Do not deploy without an explicit human response approving the resolved deployment plan.",
     "enforced_by": "human_approval.step2:sys.io.confirm"
   }

   A stronger rule for a production deployment profile could additionally state:

   {
     "rule": "When the selected deployment policy requires plan-bound evidence, the human approval response must identify the approved scope and bind it to the current effective deployment-plan digest.",
     "enforced_by": "verify_approval.step3:sys.assert"
   }

   Conceptually, the workflow would be:

   resolve the effective deployment plan
           ↓
   present the exact resolved plan to the human
           ↓
   sys.io.confirm waits for the required human-authored response
           ↓
   parse decision, approved scope, digest, signature, and other declared fields
           ↓
   verify the response deterministically
           ↓
   approve → deploy
   modify  → rebuild the plan and request confirmation again
   reject  → stop
   

   A human can include an approver identity directly in the response. If the deployment policy requires that identity or role to be authenticated rather than merely declared, the runtime or approval provider must perform that verification and return a structured result for sys.assert.

   Similarly, the response may contain:

   approver identity or role
   approved project / region / target
   approved source or image revision
   plan SHA
   public-key or key ID
   signature or signature reference
   expiry
   approval conditions
   change-ticket reference
   human comments
   

   Secret signing material should remain inside the signing provider, KMS, hardware key, or secure runtime. The human response should contain only safe references, public verification material, or generated signatures — not private keys or other secrets exposed to the model or ordinary trace output.

   Your proposed JCS + BIP-340 example would therefore fit naturally as an optional, skill-specific agents-cli deployment-evidence profile.

   For the effective deployment-plan field list, I agree that the first step is distinguishing:

   • load-bearing fields whose change alters the actual action or its risk and must therefore enter the digest; and
   • presentational fields whose wording or formatting should not invalidate the approval.

   Likely load-bearing fields include:

   deployment target
   environment
   GCP project
   region
   service name
   source revision or image digest
   Python version
   service account
   network exposure
   secret references or secret-policy digest
   resource sizing
   eval evidence
   rollback plan
   observability requirements
   

   The digest should be computed from fully resolved effective values, not merely from the raw agents-cli deploy command or unresolved defaults.

   The independent verdict should also remain semantically distinct from human authorization:

   human approval:
   the human authorizes the exact action
   
   independent verdict:
   a verifier reports whether the exact plan satisfies a particular policy
   

   They may be included in the same evidence envelope, but neither should silently substitute for the other.

   So yes — please produce the real cross-recomputable example.

   The clean framing is:

   An optional, skill-specific structured human-approval response profile implemented through sys.io.confirm, with deterministic digest and signature verification performed by the runtime or assurance tool. It extends the deployment skill without becoming a new mandatory requirement for AISP core.
9. babyblueviper1

   added a commit that references this issue on Aug 13, 2026

   Add AISP deployment-approval worked example (google/agents-cli#48)

   ...

   347f256
10. babyblueviper1 commented on Aug 13, 2026

    

    babyblueviper1

    on Aug 13, 2026

    More actions

    Real, cross-recomputable example, executed and verified before posting (not just sketched):
    https://github.com/babyblueviper1/invinoveritas/tree/main/examples/aisp-deployment-approval

    pip install coincurve && python3 deployment_approval_example.py — reproduces standalone, no dependency on anything else in that repo.

    Matches your exact shape: compute_plan_digest() JCS-canonicalizes and sha256's the load-bearing fields you listed (deployment target, region, service, revision, service account, etc.) — presentational fields like comment never enter it. build_approval_response() produces the envelope with your exact field names (decision / approver / approved_scope / plan_sha256 / key_id / expires_at), signed with real BIP-340 schnorr (same primitive Nostr/Bitcoin Taproot use). verify_approval() recomputes the digest, checks expiry, verifies the signature, returns a structured per-check result rather than a bare bool.

    Two required tests, both real, both pass: a tampered load-bearing field (region changed post-approval) correctly fails verification with plan_digest_matches: false as the named reason — an approval for us-central1 does not silently cover eu-west-1. A presentational-only edit (the comment field) correctly does NOT invalidate the approval, since it was never part of the signed digest — matching your load-bearing/presentational split applied consistently down to the envelope itself, not just the plan.

    The independent-verdict/human-approval distinction you drew holds exactly as you described in this example: verify_approval() only answers "does this signed response authorize this exact plan" — it says nothing about whether the plan itself is sound. That's a separate, composable check (a sys.assert on a policy verdict, same envelope, different signer) — didn't build that half here since it wasn't the ask, but the digest binding is identical either way, so the two compose cleanly without either substituting for the other.
11. optimization2026 commented on Aug 13, 2026

    

    optimization2026

    on Aug 13, 2026

    Author

    More actions

    Thanks — this is exactly the kind of follow-through I was hoping for: an executable example with a success case, a load-bearing-field tamper case that must fail, and a presentational-only change that must remain valid.

    The example is useful, and I would describe its current role precisely as:

    an optional deployment-approval assurance-tool proof of concept that can be called from an AISP/AISOP skill.

    It is not yet the complete AISP deployment skill or the agents-cli integration itself, which is fine. It provides the deterministic digest/signature-verification component that the full workflow can compose.

    What the example already demonstrates well

    The following parts match the intended design:

    • load-bearing deployment fields are separated from presentational fields;
    • the current plan digest is recomputed rather than trusting a producer-reported value;
    • the human-approval envelope binds a decision to a specific plan digest;
    • expiry and signature validity are checked deterministically;
    • verification returns structured per-check results rather than a bare boolean;
    • changing a load-bearing field such as region invalidates the approval;
    • changing a purely presentational field does not invalidate the approval;
    • human authorization remains distinct from an independent policy verdict.

    That is a meaningful external implementation of the approval-response shape discussed in this issue.

    How it fits into AISP/AISOP

    In the final skill, the responsibilities should remain separated:

    AISP contract
        declares the non-negotiable approval requirements
    
    AISOP flow/functions
        resolve the deployment plan
        present it to the human
        invoke sys.io.confirm
        preserve the human-authored response
        call deterministic verification
        route approve / modify / reject
    
    runtime or typed assurance tool
        performs canonicalization
        recomputes digests
        verifies signatures
        checks expiry and any authenticated identity policy
    
    sys.assert
        provides the final non-bypassable verification gate
    

    A possible contract binding remains:

    {
      "rule": "Do not deploy without a human-authored response approving the resolved deployment plan.",
      "enforced_by": "human_approval.step2:sys.io.confirm"
    }

    For deployments that require signed, plan-bound evidence, the same skill may add:

    {
      "rule": "When the selected deployment policy requires plan-bound evidence, the approval response must verify against the current effective deployment plan.",
      "enforced_by": "verify_approval.step4:sys.assert"
    }

    Conceptually:

    resolve effective deployment plan
            ↓
    present the exact plan to the human
            ↓
    sys.io.confirm waits for the required human-authored response
            ↓
    typed parser preserves and validates the response shape
            ↓
    assurance tool recomputes digest and verifies evidence
            ↓
    sys.assert checks the structured verification result
            ↓
    deploy / modify / reject
    

    The structured approval envelope may be submitted directly by the human through sys.io.confirm, or the human interaction may authorize a secure signer to produce the envelope. In either case, the runtime must not synthesize, infer, or auto-approve the human decision.

    Important next improvements

    I see several concrete improvements that would make this suitable as a reusable cross-runtime integration profile.

    1. Connect it to a real sys.io.confirm interaction

    The current script creates the key, approver value, approval response, signature, and verification inside one Python process.

    That correctly demonstrates the cryptographic envelope, but it does not yet demonstrate that the response came through a forced-blocking human interaction.

    The next integration should separate:

    plan producer
    human decision channel
    signer or approval provider
    verifier
    deployer
    

    The example can remain standalone, but the complete AISP/AISOP example should show the approval payload entering through sys.io.confirm or a runtime handler invoked by that call.

    2. Resolve the plan from actual agents-cli state

    The current resolved_plan is a hardcoded dictionary.

    For the real integration, the digest must cover the effective deployment values after all relevant sources have been resolved:

    • CLI flags;
    • agents-cli-manifest.yaml;
    • GCP project defaults;
    • region defaults;
    • deployment target;
    • service name;
    • source revision or built image digest;
    • Python version;
    • service account;
    • networking and exposure;
    • secrets configuration;
    • scaling values;
    • eval evidence;
    • rollback plan;
    • observability requirements.

    The approval should bind to what agents-cli will actually execute, not merely to the original command string or an agent-generated approximation.

    An eventual structured command such as a deployment-plan JSON output would be the cleanest integration boundary.

    3. Verify approved_scope independently

    The envelope contains both:

    approved_scope
    plan_sha256
    

    The verifier should not only check:

    compute_plan_digest(current_plan) == approval.plan_sha256
    

    It should also check:

    approved_scope == load_bearing_subset(current_plan)
    

    and:

    compute_plan_digest(approved_scope) == approval.plan_sha256
    

    Otherwise, a signed envelope could theoretically display one approved_scope while carrying a digest for another plan.

    Suggested structured checks:

    {
      "plan_digest_matches": true,
      "approved_scope_matches_current_plan": true,
      "approved_scope_digest_matches": true,
      "signature_valid": true,
      "not_expired": true,
      "decision_is_approve": true
    }

    4. Use complete RFC 8785 canonicalization or constrain the profile

    The current implementation uses:

    json.dumps(
        value,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False
    )

    For the current mostly string-based example, this is a useful deterministic representation.

    For a normative cross-language profile, however, it should either:

    • use a complete RFC 8785 JCS implementation; or
    • explicitly restrict all digest input values to a JCS-safe subset and publish fixed serialization test vectors.

    That matters for numbers, floating-point representation, Unicode edge cases, duplicate keys, negative zero, and other cross-language differences.

    BIP-340 and JCS should remain choices of this optional provider/profile, not mandatory AISP core algorithms.

    5. Distinguish key_id from the public key

    The current key_id appears to contain the x-only public key itself.

    It may be clearer to represent these separately:

    {
      "key_id": "release-key-2026-01",
      "public_key": "xonly-secp256k1:...",
      "signature_algorithm": "BIP-340",
      "signature": "..."
    }

    Alternatively, key_id can identify a key in a published registry, and the verifier can resolve the public key independently.

    6. Separate unsigned display notes from signed approval conditions

    Leaving a purely presentational comment outside the signed material is reasonable.

    But human comments sometimes contain load-bearing conditions such as:

    Deploy only during the maintenance window.
    Do not exceed ten instances.
    Approval is valid for this single execution only.
    Rollback immediately if the health check fails.
    

    Those conditions must not remain in an unsigned free-text field.

    A clearer envelope would separate:

    {
      "approval_conditions": [
        "Deploy only during the approved maintenance window",
        "Maximum instances must remain <= 10"
      ],
      "display_note": "Formatting or spelling note"
    }

    approval_conditions should enter the signed payload; display_note may remain presentational.

    7. Add domain separation and replay protection

    The signed object should identify the protocol context in which the signature is valid.

    Useful signed fields may include:

    {
      "profile": "aisp.deployment_approval_response.v1",
      "skill_id": "google_agents_cli_deploy_readiness_aisp",
      "operation": "agents-cli.deploy",
      "plan_schema": "agents-cli.effective_deployment_plan.v1",
      "audience": "agents-cli-production-deployer",
      "issued_at": "...",
      "expires_at": "...",
      "nonce": "...",
      "max_uses": 1
    }

    This reduces the risk that an approval created for one skill, operation, environment, or execution channel is reused in another context.

    8. Clarify identity claims versus authenticated identity

    A signed envelope containing:

    {
      "approver": "alice@example.com"
    }

    proves that the holder of the signing key signed a statement claiming that approver value.

    It does not automatically prove that the key belongs to Alice.

    That is not an AISP core problem, but the profile should distinguish:

    declared approver identity
    

    from:

    identity verified by a trusted runtime / IAM / KMS / approval provider
    

    If authenticated identity is required by the deployment policy, the deterministic verification result should include something like:

    {
      "identity_verified": true,
      "identity_provider": "...",
      "role_verified": true
    }

    9. Publish fixed cross-language test vectors

    The current script is executable and self-checking, but it generates a random key and time-dependent expiry on each run.

    For true cross-recomputation across independent implementations, it would be useful to add fixed fixtures:

    fixed-plan.json
    expected-canonical-plan.json
    expected-plan-sha256.txt
    fixed-private-key-test-only.txt
    fixed-public-key.txt
    fixed-approval-response.json
    expected-signature.txt
    tampered-region-plan.json
    presentational-edit-approval.json
    expected-verification-results.json
    

    Then Python, JavaScript, Rust, Go, or another implementation can verify byte-for-byte compatibility without sharing application code.

    Relationship to an independent verdict

    I agree with keeping the independent-verdict side separate.

    The two claims are:

    signed human approval:
    the human authorizes this exact deployment plan
    
    independent policy verdict:
    a verifier judges whether this exact deployment plan satisfies a policy
    

    They may use the same plan digest, but neither substitutes for the other.

    A production deployment policy may require:

    human approval verified
    AND
    independent verdict == allow
    

    while a local or development policy may require only the forced-blocking human decision.

    Current assessment

    I would now describe the example as:

    A real, executable assurance-tool proof of concept for a skill-specific structured sys.io.confirm approval response, including deterministic plan binding, BIP-340 signing, expiry checking, and a required tamper-failure test.

    It is already useful as an external implementation example.

    The next milestone is to connect it to:

    1. an actual human-authored sys.io.confirm response;
    2. a conforming AISP contract and AISOP flow;
    3. a real agents-cli effective deployment-plan resolver;
    4. deterministic sys.assert enforcement before deployment.

    Thanks for turning the discussion into running code. That is substantially more valuable than another conceptual proposal, and it gives us a concrete artifact to refine rather than an abstract interface to debate.
12. optimization2026 commented on Aug 13, 2026

    

    optimization2026

    on Aug 13, 2026

    Author

    More actions

    This is outstanding follow-through. Thank you for turning the discussion into a real, executable artifact instead of leaving it at the level of architecture diagrams and hypothetical schemas.

    I reviewed both the README and deployment_approval_example.py. The example captures the core shape remarkably well:

    • it separates load-bearing deployment fields from presentational fields;
    • it recomputes the current plan digest instead of trusting a producer-reported value;
    • it binds a human-approval envelope to a specific deployment plan;
    • it performs real BIP-340 Schnorr signing and verification;
    • it checks expiry;
    • it returns named, structured verification results rather than an opaque boolean;
    • it includes a required failure case for post-approval parameter drift;
    • and it preserves the distinction between human authorization and an independent plan-soundness verdict.

    The tamper test is particularly valuable. Demonstrating that an approval for us-central1 does not silently authorize a later change to eu-west-1 is exactly the kind of concrete failure evidence this proposal needs.

    The presentational-field test is also important. It shows that the profile is not merely hashing an entire document indiscriminately; it is attempting to define the semantic boundary between:

    changes that alter the approved action or its risk
    

    and:

    changes that only alter presentation
    

    That is substantially stronger than another conceptual proposal. It gives us a real object to inspect, run, challenge, and improve.

    How I would classify the example

    At its current stage, I would describe it as:

    A real, executable deployment-approval assurance-tool proof of concept for a skill-specific structured sys.io.confirm response.

    That is already a meaningful result.

    It is not yet the complete AISP skill or the complete agents-cli integration, but it provides the deterministic evidence component that such a skill can call.

    The intended composition would be:

    AISP contract
            ↓
    AISOP execution flow
            ↓
    resolve effective agents-cli deployment plan
            ↓
    sys.io.confirm
            ↓
    human-authored structured approval response
            ↓
    this digest/signature verification tool
            ↓
    sys.assert
            ↓
    agents-cli deploy
    

    The current example implements the approval-envelope construction and verification portion of that chain very well.

    AISP/AISOP integration boundary

    The final integration should keep the responsibilities clear.

    AISP contract

    The contract declares the non-negotiable approval requirement:

    {
      "rule": "Do not deploy without a human-authored response approving the resolved deployment plan.",
      "enforced_by": "human_approval.step2:sys.io.confirm"
    }

    A production deployment profile may add:

    {
      "rule": "When the selected deployment policy requires plan-bound evidence, the approval response must verify against the current effective deployment plan.",
      "enforced_by": "verify_approval.step4:sys.assert"
    }

    AISOP flow and functions

    The executable workflow should:

    • resolve the effective deployment plan;
    • present the exact resolved values to the human;
    • block at sys.io.confirm;
    • preserve the human-authored response;
    • route approve, modify, and reject;
    • invoke deterministic evidence verification;
    • and prevent deployment unless the required checks pass.

    Runtime or typed assurance tool

    The runtime/tool should perform:

    • canonicalization;
    • digest computation and recomputation;
    • signature verification;
    • expiry checking;
    • identity or role verification when required;
    • and structured failure reporting.

    sys.assert

    The final deployment gate should consume the structured result rather than asking the LLM to judge whether the evidence “looks valid.”

    Conceptually:

    sys.assert(
      "approval_verification.valid == true",
      "The deployment approval does not authorize the current effective plan"
    )
    

    The most important next step: real human interaction

    The current Python example creates the signing key, approver value, approval response, signature, and verification result inside one process.

    That is completely reasonable for demonstrating the cryptographic construction.

    The next milestone is to connect that construction to an actual forced-blocking human response:

    present plan
        ↓
    sys.io.confirm blocks
        ↓
    human provides or authorizes the structured response
        ↓
    secure signer produces the signature
        ↓
    verifier checks the evidence
    

    The AI must not synthesize, infer, complete, or approve the response for the human.

    The signing key should remain inside a secure signer, KMS, hardware key, approval provider, or equivalent runtime boundary. The model should only receive safe public material, signatures, or references.

    Resolve the plan from real agents-cli state

    The example currently uses a hardcoded resolved_plan, which is appropriate for a standalone PoC.

    For the actual integration, the approved object must represent what agents-cli will really execute after resolving:

    • CLI flags;
    • agents-cli-manifest.yaml;
    • GCP project defaults;
    • deployment target;
    • region;
    • service name;
    • source revision or image digest;
    • Python version;
    • service account;
    • network exposure;
    • secrets configuration;
    • resource sizing;
    • eval evidence;
    • rollback plan;
    • and observability requirements.

    The approval should bind to the effective deployment plan, not merely to the raw command:

    agents-cli deploy
    

    An eventual structured interface such as:

    agents-cli deploy --plan-json
    

    would be the cleanest integration boundary, although an external adapter can prototype the same shape first.

    One important consistency check

    The envelope contains both:

    approved_scope
    

    and:

    plan_sha256
    

    The verifier currently recomputes the digest from the current plan and compares it with approval.plan_sha256.

    I recommend also verifying the relationship between approved_scope and that digest explicitly:

    approved_scope == load_bearing_subset(current_plan)
    

    and:

    compute_plan_digest(approved_scope) == plan_sha256
    

    That would produce checks such as:

    {
      "plan_digest_matches": true,
      "approved_scope_matches_current_plan": true,
      "approved_scope_digest_matches": true,
      "signature_valid": true,
      "not_expired": true,
      "decision_is_approve": true
    }

    This prevents a malformed or malicious signer from displaying one approved scope while attaching a digest for another plan.

    Required-field validation

    The current use of resolved_plan.get(field) means a missing load-bearing field becomes null and can still enter the digest.

    For a deployment-readiness profile, some fields should fail closed when missing.

    For example, production deployment may require:

    deployment_target
    gcp_project
    region
    service_name
    source_revision or image_digest
    service_account
    rollback_plan
    

    The profile should define which fields are:

    • required;
    • conditionally required;
    • optional;
    • or mutually exclusive.

    That validation should happen before approval is requested.

    Canonicalization

    The current serializer is a useful deterministic implementation for the example's mostly string-based data:

    json.dumps(
        value,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False
    )

    For a normative cross-language profile, I suggest either:

    1. using a complete RFC 8785 JCS implementation; or
    2. explicitly constraining the digest input to a documented JCS-safe data subset.

    The distinction matters for floating-point values, number serialization, negative zero, Unicode edge cases, duplicate properties, and other language-dependent behavior.

    The current example demonstrates the architecture correctly; the next step is to make the canonical byte representation normative and independently reproducible.

    key_id and public key

    The current key_id appears to carry the x-only public key directly.

    For a reusable profile, it may be clearer to separate:

    {
      "key_id": "release-key-2026-01",
      "public_key": "xonly-secp256k1:...",
      "signature_algorithm": "BIP-340",
      "signature": "..."
    }

    Alternatively, key_id can refer to a published key registry from which the verifier resolves the public key.

    BIP-340 is a valid choice for this provider/profile, but it should remain an optional implementation choice rather than becoming a mandatory AISP core algorithm.

    Signed conditions versus presentational notes

    Excluding a purely presentational comment from the signed payload is sensible.

    However, approval comments can sometimes contain action-changing conditions:

    Deploy only during the maintenance window.
    Do not exceed ten instances.
    This approval is valid for one execution only.
    Rollback immediately if the health check fails.
    

    Those conditions are load-bearing and must be signed.

    A clearer envelope could separate:

    {
      "approval_conditions": [
        "Deploy only during the approved maintenance window",
        "Maximum instances must remain <= 10"
      ],
      "display_note": "Formatting-only or explanatory note"
    }

    approval_conditions should enter the signed payload. display_note may remain presentational.

    Domain separation and replay protection

    For a production profile, the signed payload should identify the context in which it is valid.

    Useful fields could include:

    {
      "profile": "aisp.deployment_approval_response.v1",
      "skill_id": "google_agents_cli_deploy_readiness_aisp",
      "operation": "agents-cli.deploy",
      "plan_schema": "agents-cli.effective_deployment_plan.v1",
      "audience": "agents-cli-production-deployer",
      "issued_at": "...",
      "expires_at": "...",
      "nonce": "...",
      "max_uses": 1
    }

    This helps prevent a valid approval from being replayed:

    • for another skill;
    • for another operation;
    • in another environment;
    • or for multiple executions when only one was authorized.

    Identity claim versus authenticated identity

    A signed envelope containing:

    {
      "approver": "alice@example.com"
    }

    proves that the signing key signed a statement containing that approver claim.

    It does not by itself prove that the key belongs to Alice or that Alice has deployment authority.

    That is not an AISP core requirement, but the optional profile should distinguish:

    declared approver
    

    from:

    authenticated and authorized approver
    

    When authentication is required, the verifier can return additional structured fields:

    {
      "identity_verified": true,
      "identity_provider": "example-idp",
      "role_verified": true,
      "verified_role": "release-manager"
    }

    The deployment skill can then decide whether those fields are required by policy.

    Fixed cross-language test vectors

    The current script is executable and self-checking, which is excellent.

    To make “cross-recomputable” testable across independent implementations, I recommend adding fixed fixtures:

    fixed-plan.json
    expected-canonical-plan.bin
    expected-plan-sha256.txt
    fixed-public-key.txt
    fixed-approval-response.json
    expected-signature.txt
    tampered-region-plan.json
    presentational-edit-response.json
    expected-verification-results.json
    

    A Python, JavaScript, Rust, Go, or Java implementation should then produce the same digest and verification results from the same fixtures.

    That would turn the example from a strong Python PoC into a genuine interoperability profile.

    Independent verdict remains separate

    I strongly agree with the decision not to fold the independent verdict into approval verification.

    The two claims remain:

    human approval:
    a human authorizes this exact action
    
    independent verdict:
    a verifier reports whether this exact plan satisfies a policy
    

    They can bind to the same plan digest, but neither substitutes for the other.

    A production policy may require:

    human approval verified
    AND
    independent verdict == allow
    

    A development environment may require only the non-bypassable human response.

    Overall assessment

    This is an excellent contribution.

    It demonstrates that the structured approval-response model discussed here is not only expressible but implementable, testable, and composable.

    The example has already moved the discussion forward in a substantial way:

    from:
    a prose-only human-approval instruction
    
    to:
    a forced human response point
    plus a structured, plan-bound, cryptographically verifiable approval envelope
    

    The next milestone is not more abstract discussion. It is connecting this assurance tool to:

    1. a real sys.io.confirm human response;
    2. a conforming AISP contract and AISOP flow;
    3. an actual agents-cli effective deployment-plan resolver;
    4. deterministic sys.assert enforcement;
    5. and fixed cross-language test vectors.

    Thank you again for investing the time to produce a real implementation, including the required failure case. This is exactly how protocol and runtime ideas become credible engineering artifacts.
13. babyblueviper1

    added a commit that references this issue on Aug 13, 2026

    v2 of AISP deployment-approval example: implement optimization2026's …

    ...

    be02239
14. babyblueviper1 commented on Aug 13, 2026

    

    babyblueviper1

    on Aug 13, 2026

    More actions

    v2, real and executed before posting: https://github.com/babyblueviper1/invinoveritas/tree/main/examples/aisp-deployment-approval

    Implemented the self-contained items from your review list:

    • (3) approved_scope verified independently of plan_sha256 — added approved_scope_matches_current_plan + approved_scope_digest_matches as separate checks, plus a dedicated scope-forgery test: editing approved_scope after signing now breaks the signature itself (it's inside the signed payload, not just implied by the plan digest), so a malformed signer can't display one scope while a digest secretly covers another plan.
    • (5) key_id split from public_key — key_id is now an opaque registry label, public_key is the actual x-only hex.
    • (6) signed approval_conditions split from unsigned display_note — "deploy only in the maintenance window" now enters the signature; a note typo fix doesn't.
    • (7) domain separation / replay protection — profile/skill_id/operation/plan_schema/audience/issued_at/nonce/max_uses are all signed. Added a replay test showing a reused nonce fail on its second verify — named plainly as an in-memory demo guard, not a claim that this alone solves production replay protection (needs durable shared storage across verifier instances).
    • Required-field fail-closed validation, and canonicalization pinned to your named fallback (a documented, enforced JCS-safe subset — no float, no bool-as-int) rather than pulling in a full RFC 8785 dependency casually.
    • (9) fixed cross-language --fixtures — committed under vectors/.

    One real bug this surfaced, worth naming since it's exactly the kind of thing "cross-recomputable" is supposed to catch: BIP-340 Schnorr signing mixes in randomized auxiliary data by default, so my first --fixtures pass silently produced a different signature every run despite an identical key and message — which would have made the "byte-for-byte reproducible" claim false on inspection. Fixed with an explicit fixed aux_randomness in the fixtures path only; verified by running --fixtures twice into separate directories and diffing (identical, including the signature bytes).

    Still not attempted, same three as before — these need your side to be real, not something a standalone example can honestly claim: wiring an actual sys.io.confirm human-response channel, resolving the plan from real agents-cli state, and identity authentication against an IdP.
15. optimization2026 commented on Aug 13, 2026

    

    optimization2026

    on Aug 13, 2026

    Author

    More actions

    This is exceptional engineering follow-through.

    Thank you for responding to the review with a real v2 implementation rather than another layer of design prose. The speed, precision, and honesty of this iteration are genuinely impressive.

    You implemented nearly every self-contained item from the review:

    • independent approved_scope verification;
    • a dedicated scope-forgery failure case;
    • a clean key_id / public_key separation;
    • signed approval_conditions separated from an unsigned display_note;
    • signed domain-separation and replay-context fields;
    • an explicit replay test with an honestly documented in-memory limitation;
    • required-field fail-closed validation;
    • a deliberately constrained and enforced canonicalization subset rather than an overstated RFC 8785 claim;
    • and fixed cross-language fixtures.

    The BIP-340 auxiliary-randomness bug you found while building the fixtures is especially valuable.

    That is exactly what executable interoperability vectors are supposed to do: expose a claim that appears correct at the architectural level but is false at the byte level. Catching that identical key/message inputs did not produce identical fixture signatures, understanding why, and confining fixed aux_randomness to the test-vector path while retaining real randomness for normal execution is excellent engineering discipline.

    Just as importantly, you did not overclaim the parts that the standalone example cannot honestly prove. Explicitly naming the boundary around durable replay state, real agents-cli resolution, and identity authentication makes the implementation much more credible.

    One correction from my previous review

    I previously described “wiring an actual sys.io.confirm human-response channel” as a missing runtime component.

    That was too broad.

    It is missing from the standalone v2 example, but the human-response Runtime already exists in SoulBot.

    References:

    • SoulBot repository:
      https://github.com/AIXP-Labs/SoulBot

    • SoulBot Execute Engine example:
      https://github.com/AIXP-Labs/SoulBot/tree/main/examples/simple/Soul_Agent/soulbot_execute_engine_aiap

    • Agent Engine — executes sys.io.confirm, writes WAITING_USER, and resumes using USER_ANSWER:
      https://github.com/AIXP-Labs/SoulBot/blob/main/examples/simple/Soul_Agent/soulbot_execute_engine_aiap/agent_engine.aisop.json

    • Router — presents the pending question, waits for the complete human response, and re-dispatches the same node:
      https://github.com/AIXP-Labs/SoulBot/blob/main/examples/simple/Soul_Agent/soulbot_execute_engine_aiap/main.aisop.json

    • Node Engine — preserves the raw user response, resumes the same node, and audits user-gate execution:
      https://github.com/AIXP-Labs/SoulBot/blob/main/examples/simple/Soul_Agent/soulbot_execute_engine_aiap/node_engine.aisop.json

    • Mechanical user-gate audit and WAITING_USER coercion backstop:
      https://github.com/AIXP-Labs/SoulBot/blob/main/examples/simple/Soul_Agent/soulbot_execute_engine_aiap/python_tools/user_gate_audit.py

    The implemented Runtime path is:

    AISOP node reaches sys.io.confirm
            ↓
    Agent Engine writes WAITING_USER and stops
            ↓
    Router presents the question to the human
            ↓
    Router waits for the complete human response
            ↓
    the same node is re-dispatched with:
    
    RESUME_MODE=true
    USER_ANSWER={raw user response}
    
            ↓
    Agent Engine reads the complete USER_ANSWER
            ↓
    the resumed node continues from steps_remaining
    

    The Runtime explicitly forbids:

    auto-approval
    silent bypass
    batching the confirmation away
    AI-generated substitution for the human response
    

    The Node Engine also records the raw human user_answer and may record parsed accepted and rejected items for audit and resume handling.

    In addition, user_gate_audit.py --enforce is a mechanical backstop: if a node that required a human gate reaches a terminal state without presenting that gate, the tool can coerce it back to WAITING_USER and require the human decision again.

    So the remaining work is not to invent a human-response Runtime.

    The remaining work is to connect your v2 approval verifier to that already-existing Runtime path.

    options=['approve', 'reject', 'modify'] is sufficient

    No new mandatory response_schema parameter or new AISP contract field is required for this integration.

    The existing interaction can use:

    options=['approve', 'reject', 'modify']
    

    as the stable decision vocabulary.

    Those options define the control-flow decision, not the maximum amount of information the human is allowed to provide.

    A human response may therefore be:

    approve {
      "approver": "alice@example.com",
      "approved_scope": {
        "deployment_target": "agent_runtime",
        "gcp_project": "company-prod",
        "region": "us-central1",
        "service_name": "support-agent",
        "source_revision": "abc123",
        "python_version": "3.13"
      },
      "plan_sha256": "sha256:...",
      "key_id": "release-key-2026-01",
      "public_key": "...",
      "signature": "...",
      "expires_at": 1786003600,
      "approval_conditions": [
        "Deploy only during the approved maintenance window"
      ]
    }
    

    Or:

    modify {
      "region": "europe-west1",
      "reason": "Use the approved data-residency region"
    }
    

    Or:

    reject {
      "reason": "The eval evidence is stale"
    }
    

    The leading token provides the routing anchor:

    approve
    reject
    modify
    

    The remainder is skill-defined human-authored content.

    SoulBot preserves the complete response as USER_ANSWER, so the resumed LLM can understand both the decision and the attached information.

    The LLM can then:

    1. identify the leading decision;
    2. preserve the attached approval payload;
    3. call the appropriate deterministic tool;
    4. route the workflow according to the decision.

    The important boundary is that the LLM should not itself decide that a signature, digest, or expiry “looks valid.”

    The model performs semantic interpretation and tool orchestration.

    Your v2 verifier performs the deterministic evidence checks.

    sys.assert provides the final hard gate.

    Correct AISP/AISOP responsibility split

    The resulting architecture is:

    AISP contract
        declares the non-negotiable human-approval red line
    
    AISOP function
        presents the resolved plan
        invokes sys.io.confirm
        defines approve / reject / modify semantics
    
    SoulBot Runtime
        forces WAITING_USER
        waits for the real human response
        preserves and returns the complete USER_ANSWER
        resumes the same node
    
    LLM
        understands the decision and attached content
        calls the verification tool
        does not perform cryptographic judgment itself
    
    v2 assurance tool
        validates scope
        recomputes the plan digest
        validates the signature
        checks expiry and domain context
        checks replay policy where configured
    
    sys.assert
        prevents deployment unless deterministic verification passes
    

    This is exactly the AISP division of labor:

    Tier S:
    declare the red line and bind it to a real mechanism
    
    Tier A:
    define the interaction, interpretation, tool invocation,
    verification, branching, and failure behavior
    
    Runtime:
    make sys.io.confirm and sys.assert real
    

    Contract binding

    The baseline human-sovereignty rule can be:

    {
      "rule": "Do not deploy without an explicit human decision on the resolved deployment plan.",
      "enforced_by": "human_approval.step2:sys.io.confirm"
    }

    A production profile using your evidence mechanism can additionally declare:

    {
      "rule": "When the human decision is approve, deployment must not proceed unless the attached approval evidence validates against the current effective deployment plan.",
      "enforced_by": "verify_approval.step2:sys.assert"
    }

    Conceptually, the AISOP functions can use this shape:

    {
      "human_approval": {
        "step1": "Present the complete effective deployment plan, including target, project, region, source or image revision, eval evidence, IAM/secrets status, rollback plan, and observability requirements.",
        "step2": "sys.io.confirm('Reply with approve {approval evidence}, reject {reason}, or modify {requested changes}', options=['approve', 'reject', 'modify']) -> human_response",
        "step3": "Interpret the leading human decision and preserve the remaining human-authored content without inventing, completing, or silently altering approval evidence.",
        "output_mapping": "human_decision"
      },
      "verify_approval": {
        "step1": "When human_decision is approve, call the deterministic approval-verification tool with the current effective deployment plan and the preserved approval payload.",
        "step2": "sys.assert('human_decision.decision != \"approve\" || approval_verification.valid == true', 'The human approval evidence does not authorize the current effective deployment plan')",
        "output_mapping": "approval_verification"
      }
    }

    The control flow is then:

    graph TD
        resolve_plan[Resolve effective deployment plan]
            --> human_approval[Wait for human decision]
    
        human_approval --> decision{Human decision}
    
        decision -->|approve| verify_approval[Verify approval evidence]
        decision -->|modify| modify_plan[Apply requested changes]
        decision -->|reject| cancelled[Stop without deployment]
    
        modify_plan --> resolve_plan
        verify_approval --> verified{Evidence valid?}
    
        verified -->|yes| deploy[Run agents-cli deploy]
        verified -->|no| blocked[Block deployment]
    

    Loading

    A modified plan necessarily returns to resolve_plan and then to a new human confirmation. Approval for the previous plan must never silently carry over.

    Preserve the raw human evidence

    The LLM can understand and route a response such as:

    approve {...}
    

    but the cryptographic verifier should receive the raw approval payload, or a losslessly extracted equivalent, rather than an LLM paraphrase.

    This matters because signatures, hashes, key material, nonces, and exact field values are not natural-language content. A model may understand their purpose, but it must not rewrite them before verification.

    The clean pattern is:

    raw USER_ANSWER
            ↓
    LLM identifies approve / reject / modify
            ↓
    exact approval payload passed to verification tool
            ↓
    structured verification result
            ↓
    sys.assert
    

    What remains genuinely open

    After correcting the runtime point, the remaining integration work is narrower and clearer.

    1. Connect v2 to the existing SoulBot human gate

    The v2 implementation can become:

    approval.verify
    

    or an equivalent typed Runtime Tool invoked by the resumed AISP node.

    The standalone script does not need to become the human channel itself.

    2. Resolve the effective plan from real agents-cli state

    This remains the largest genuine integration requirement.

    The plan must reflect the actual values that agents-cli will execute after resolving:

    CLI flags
    manifest values
    GCP project defaults
    region defaults
    deployment target
    service name
    source revision or image digest
    Python version
    service account
    network exposure
    secret configuration
    resource sizing
    eval evidence
    rollback plan
    observability requirements
    

    The human must approve the effective plan, not a loose command string or an LLM approximation.

    A native agents-cli plan-output interface would be ideal, but an adapter can prototype it first.

    3. Package the workflow as a conforming AISP skill

    The final package should contain:

    google_agents_cli_deploy_readiness_aisp/
    ├── aisp.aisop.json
    ├── README.md
    ├── SKILL.md
    ├── scripts/
    │   └── approval_verifier.py
    ├── schemas/
    └── evals/
        └── vectors/
    

    The v2 script and vectors would be declared resources or exposed through a typed Runtime capability.

    4. Identity authentication remains optional

    An IdP is not required for baseline AISP Human Sovereignty.

    A human can validly approve through the forced-blocking channel without AISP globally defining who that human is.

    If a particular production policy requires:

    authenticated identity
    release-manager role
    multiple approvers
    KMS-backed signing
    

    that requirement can be declared by this deployment skill and enforced by its Runtime or approval provider.

    It is an optional higher-assurance deployment policy, not a missing AISP core primitive.

    Final assessment

    Your v2 is an outstanding contribution.

    It is not merely an illustrative snippet anymore. It is a serious, testable assurance component with:

    explicit scope
    real cryptography
    named verification checks
    negative tests
    domain separation
    replay semantics
    fail-closed validation
    fixed interoperability vectors
    and honestly documented boundaries
    

    The fact that you found a real nondeterminism defect while producing the fixtures is particularly strong evidence that the exercise was substantive rather than performative.

    And your refusal to pretend that a standalone script had implemented host-dependent behavior it had not implemented is equally important. That kind of precision is exactly what makes security and protocol work trustworthy.

    The only correction is that the real human-response channel already exists in SoulBot. What remains is integration:

    existing SoulBot sys.io.confirm Runtime
            +
    your v2 deterministic assurance tool
            +
    a real agents-cli effective-plan resolver
            +
    a conforming AISP skill
    

    That is now a concrete and achievable engineering path.

    Thank you again for taking the review seriously enough to turn it into a stronger implementation within hours. This is one of the clearest examples of protocol discussion becoming executable, falsifiable engineering evidence.
16. babyblueviper1 commented on Aug 13, 2026

    

    babyblueviper1

    on Aug 13, 2026

    More actions

    Checked SoulBot directly before replying (not taking the citation on faith) — real repo, and agent_engine.aisop.json genuinely has the WAITING_USER/sys.io.confirm mechanism you described, user_gate_audit.py is really there under python_tools/. Correction accepted — the human-response Runtime isn't missing, it's a wiring problem, which is a much better place to be than "doesn't exist yet."

    The responsibility split you laid out is right, and the part worth restating back because it's the one a lazier integration would get wrong: the LLM identifies the decision and routes, it never adjudicates the evidence. USER_ANSWER passed raw to the verifier, not re-typed by a model in the middle, is the same discipline as the plan digest itself — anything that goes through a paraphrase step stops being independently recomputable. Good catch naming it explicitly rather than leaving it implied by the architecture diagram.

    options=['approve','reject','modify'] as the routing anchor with the full evidence payload riding in the response body is cleaner than what I'd have reached for (a typed response_schema) — no new AISP surface, and it composes with what v2 already emits.

    On the two things you named as genuinely open: the agents-cli real-plan resolver is the one that actually gates whether this is real or a demo, and it's not something I can honestly scope from outside your CLI's internals. Packaging as a conforming AISP skill is more tractable on my end once the resolver exists to point it at. Not committing to a timeline here since both are real work, not a same-day follow-up — but this is now a concrete, load-bearing integration target rather than a standalone example, which is the actual milestone from where this started three days ago.
17. babyblueviper1

    added a commit that references this issue on Aug 13, 2026

    AISP v3: real agents-cli plan resolver + conforming skill package (go…

    ...

    5897ff8
18. babyblueviper1 commented on Aug 13, 2026

    

    babyblueviper1

    on Aug 13, 2026

    More actions

    Built both, real and executed: https://github.com/babyblueviper1/invinoveritas/tree/main/examples/aisp-deployment-approval

    resolve_agents_cli_plan.py — against a fresh clone of google/agents-cli, not guessed. Read cmd_deploy.py's real click options and the actual agents-cli-manifest.yaml template. CLI-flag-over-manifest precedence matches what cmd_deploy.py itself does (cfg.deployment_target = deployment_target when the flag is set; region = region or cfg.region; resolve_service_name).

    Honest coverage, not a wishful mapping:

    • Mapped: deployment_target, gcp_project, region, service_name, service_account, resource_sizing, network_exposure (assembled from --iap/--network-attachment/--dns-peering-*), secret_policy_digest (sha256 of secret identifiers from --secrets, never values).
    • Partial: source_revision — only when --image is passed; from-source deploys have no first-class revision in agents-cli at all.
    • Honestly absent: environment, python_version, eval_evidence, rollback_plan, observability_requirements — none have a real agents-cli source today. Worth naming specifically: agents-cli has no rollback command at all, so a plan resolved purely from its own state is correctly refused by validate_plan_completeness until a human or policy supplies rollback_plan out of band. That's the fail-closed design working as intended, not a gap in the resolver.
    • --supplements can only fill fields the resolver left absent — it errors if asked to override anything actually mapped, so a supplement can't launder a fake value past a real resolved one.

    google_agents_cli_deploy_readiness_aisp/ — the conforming skill package in the exact shape you specified. aisp.aisop.json's human_approval/verify_approval steps and the two non_negotiable rules are your exact text from this thread, not paraphrased — diffed them against your comment before committing. scripts/approval_verifier.py composes the resolver with the already-tested v2 verify functions (imports, no duplicated crypto). evals/vectors/ symlinks the existing fixed fixtures rather than regenerating them.

    Ran the full composition myself before shipping: resolve a real-shaped plan → refused for missing rollback_plan → supplement only the honestly-absent fields → build → verify (valid: true, all 7 checks) → tamper region → correctly fails on both plan_digest_matches and approved_scope_matches_current_plan → confirmed a supplement can't override a mapped field either.

    Not attempted, same as before: no PR against agents-cli itself (not our repo), no SoulBot Runtime execution, no IdP identity binding — those stay wiring/optional policy per your own scoping.
19. optimization2026 commented on Aug 15, 2026

    

    optimization2026

    on Aug 15, 2026

    Author

    More actions

    This is an exceptional piece of follow-through.

    You did not merely respond to the architecture discussion. You independently inspected the SoulBot Runtime, corrected your own model of the missing integration boundary, read the real google/agents-cli deployment implementation, built an effective-plan resolver from that source, composed it with the signed-approval verifier, and then packaged the result into an AISP-shaped skill candidate.

    That is an extraordinary amount of serious engineering progress in a remarkably short period of time.

    The progression is now:

    initial trust-boundary question
            ↓
    executable signed-approval proof of concept
            ↓
    fixed interoperability vectors
            ↓
    independent SoulBot Runtime verification
            ↓
    real agents-cli source analysis
            ↓
    effective-plan resolver
            ↓
    AISP contract + AISOP flow + schemas + verifier composition
    

    This is no longer just an interesting standalone cryptographic example.

    I would now describe it as:

    A real, source-grounded AISP deployment-approval candidate integration for agents-cli.

    That is a major milestone.

    What is especially strong

    1. You verified SoulBot rather than accepting the citation on trust

    This matters.

    You independently confirmed that SoulBot actually implements:

    sys.io.confirm
            ↓
    WAITING_USER
            ↓
    raw USER_ANSWER
            ↓
    same-node resume
    

    and that user_gate_audit.py exists as a mechanical backstop.

    That turns the Human Sovereignty path from an architectural assertion into an independently inspected implementation fact.

    Your correction is exactly right:

    The human-response Runtime is not missing. The remaining problem is wiring the verifier into that Runtime.

    SoulBot references:

    • https://github.com/AIXP-Labs/SoulBot
    • https://github.com/AIXP-Labs/SoulBot/tree/main/examples/simple/Soul_Agent/soulbot_execute_engine_aiap

    2. You preserved the correct LLM/tool boundary

    Your restatement is the most important integration rule:

    The LLM identifies the decision and routes; it never adjudicates the evidence.

    That is exactly the right boundary.

    LLM:
    understands approve / reject / modify
    preserves the attached payload
    selects the next workflow action
    calls the verifier
    
    deterministic verifier:
    recomputes the digest
    checks scope
    checks signature
    checks expiry
    checks domain context
    returns named verification results
    
    sys.assert:
    provides the final deployment gate
    

    Passing the raw USER_ANSWER payload into the verifier, rather than reconstructing or paraphrasing it through the model, is essential. Once signed fields pass through a lossy model rewrite, independent recomputation is no longer reliable.

    Your explicit recognition of that point substantially strengthens the integration.

    3. The resolver is based on real agents-cli source

    The new resolve_agents_cli_plan.py is a meaningful advance because it does not invent an imaginary deployment schema.

    You inspected:

    • the current cmd_deploy.py options;
    • the real agents-cli-manifest.yaml shape;
    • CLI-over-manifest precedence;
    • target-specific flag restrictions;
    • service-name resolution;
    • resource defaults;
    • networking options;
    • Secret Manager references;
    • and the actual gaps in the deploy command.

    The mapped / partial / absent distinction is particularly good:

    mapped:
    a real source exists
    
    partial:
    a real source exists, but it does not represent the full intended meaning
    
    absent:
    agents-cli has no first-class source, so the resolver refuses to fabricate one
    

    That is exactly the kind of honesty an execution contract needs.

    4. The missing rollback capability was surfaced rather than hidden

    Calling out that agents-cli currently has no first-class rollback command or structured rollback-plan field is very valuable.

    Instead of quietly inserting a plausible rollback string and pretending it came from the CLI, the resolver leaves the field absent and allows production completeness checks to fail closed.

    That is not a weakness in the resolver.

    That is the resolver doing its job correctly:

    missing real capability
            ≠
    permission for the model to invent one
    

    This is one of the clearest demonstrations of why AISP-style contracts are useful: unsupported assumptions become explicit state rather than invisible model improvisation.

    5. Supplements cannot overwrite resolved values

    The rule that --supplements may fill only honestly absent fields, and may not overwrite fields already resolved from real agents-cli inputs, is a strong safeguard.

    It prevents an external policy layer from laundering a different value over a real CLI-derived value.

    The demonstrated sequence is meaningful:

    resolve real-shaped plan
            ↓
    refuse incomplete production plan
            ↓
    supplement only honestly absent fields
            ↓
    build approval
            ↓
    verify successfully
            ↓
    tamper region
            ↓
    fail digest and scope verification
    

    That is a real composition test, not an isolated unit demonstration.

    6. You packaged the result instead of stopping at an adapter

    The addition of:

    google_agents_cli_deploy_readiness_aisp/
    ├── aisp.aisop.json
    ├── README.md
    ├── SKILL.md
    ├── scripts/
    ├── schemas/
    └── evals/
    

    is a major step.

    The two non_negotiable rules are also correctly focused:

    {
      "rule": "Do not deploy without an explicit human decision on the resolved deployment plan.",
      "enforced_by": "human_approval.step2:sys.io.confirm"
    }

    {
      "rule": "When the human decision is approve, deployment must not proceed unless the attached approval evidence validates against the current effective deployment plan.",
      "enforced_by": "verify_approval.step2:sys.assert"
    }

    This is the correct AISP separation:

    Tier S:
    declare the red lines
    
    Tier A:
    define resolution, human interaction, verification,
    branching, modification, rejection, and deployment
    
    Runtime:
    make sys.io.confirm, tools, state, and sys.assert real
    

    The implementation has now crossed an important boundary: an external developer has used AISP not just as terminology, but as an actual engineering structure.

    One important refinement: this is a candidate integration, not yet the final effective-plan boundary

    The resolver is real and source-grounded, but it currently represents a projection of agents-cli deployment behavior rather than the entire action-affecting surface.

    That distinction matters because an approval digest is only as strong as the set of execution inputs it commits to.

    Some currently recognized but unmapped CLI inputs can materially change the deployed result:

    --update-env-vars
    --agent-identity
    --port
    --build-args
    --cluster-name
    --no-wait
    

    The runtime deployment behavior also depends on values such as:

    .env content
    AGENT_VERSION
    ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS
    create-versus-update state
    live Cloud Run sizing on update
    generated source-build output
    Terraform-controlled GKE state
    

    These are not merely presentational fields.

    For example:

    approval plan:
    UPDATE_ENV FEATURE_FLAG=false
    
    actual deployment:
    UPDATE_ENV FEATURE_FLAG=true
    

    must not produce the same approved action digest.

    Likewise, changing:

    agent identity
    container port
    build args
    GKE cluster
    environment variables
    

    can change the deployed system without changing the currently defined plan digest.

    So I would make the next resolver rule:

    Every input that can materially change the deployed artifact, execution target, permissions, network exposure, runtime behavior, or completion semantics must either enter the effective-plan digest or be deterministically prohibited by the profile.

    Operational flags such as:

    --list
    --status
    --dry-run
    

    should probably route to separate non-deployment operations rather than enter the deployment-approval path.

    Supplements should be divided into two categories

    The current absent-only rule is good, but the next version should distinguish:

    Policy or evidence supplements

    These can legitimately come from outside agents-cli:

    eval_evidence
    rollback_plan
    observability_requirements
    environment classification
    change-ticket or governance references
    

    They describe why the deployment may be allowed.

    Execution inputs

    These determine what is actually executed:

    gcp_project
    service_account
    source or image identity
    python/runtime version
    port
    build arguments
    cluster name
    environment variables
    resource sizing
    network configuration
    

    An execution input cannot safely exist only inside the approval plan.

    If a supplement supplies or changes an execution input, that value must also become an input to the actual deployment command or typed deployment tool.

    Otherwise the system can approve:

    service_account = A
    

    while the unchanged CLI invocation actually deploys with:

    platform-default service account B
    

    The strongest boundary would be:

    resolve effective plan
            ↓
    approve effective plan
            ↓
    deploy consumes that same resolved plan
    

    rather than:

    resolve from flags
            ↓
    approve projection
            ↓
    run the original flags and resolve again independently
    

    A native future interface such as:

    agents-cli deploy --plan-json
    

    would be ideal, but the adapter can prototype the same invariant first.

    Production mode should prefer explicit, immutable inputs

    For production assurance, I would strongly consider requiring:

    explicit --project
    explicit service account or an independently resolved platform identity
    immutable image digest or source-tree digest
    explicit deployment target
    explicit region
    

    The current resolver correctly refuses to call ADC or gcloud config, but allowing an out-of-band gcp_project supplement while later invoking the original command without --project could create a mismatch:

    approved project:
    company-prod
    
    actual project resolved by agents-cli:
    whatever gcloud config currently contains
    

    Similarly, a supplemented Git revision does not prove that a source deployment from . uses that revision. The working tree, Dockerfile, dependency lock, uncommitted changes, and build context can all differ.

    For high-assurance source deployment, the approval should bind to something such as:

    source-tree digest
    build-context digest
    or immutable produced image digest
    

    not merely a human-supplied Git label.

    AISP package self-containment needs one more pass

    The current directory is an excellent candidate package, but I would reserve the strict word conforming until official AISP validation evidence is posted.

    The main packaging concern is that:

    scripts/approval_verifier.py
    

    imports sibling modules outside the Skill Folder:

    ../deployment_approval_example.py
    ../resolve_agents_cli_plan.py
    

    and:

    evals/vectors
    

    is a symlink to a directory outside the Skill Folder.

    That means the folder is not currently a self-contained source of truth when copied, registered, hashed, zipped, or moved independently.

    A portable AISP package should either:

    1. move the resolver, verifier implementation, and vectors inside the Skill Folder; or
    2. place reusable assets under the AISP _shared/ boundary and declare them as shared resources.

    The resolved path of a declared resource should not escape the package through a symlink.

    This is not cosmetic. It affects:

    portability
    package hashing
    resource inventory
    registry publication
    Windows compatibility
    archive extraction
    supply-chain review
    

    The AISOP tool result should be bound explicitly

    The intent of verify_approval is correct, but the executable binding should be more explicit.

    Current conceptual shape:

    step1:
    Call the deterministic approval-verification tool
    
    step2:
    sys.assert(approval_verification.valid == true)
    

    The Runtime still needs an unambiguous machine binding showing how:

    approval_verification
    

    is produced.

    For example:

    sys.run(
      'python scripts/approval_verifier.py verify ...'
    ) -> approval_verification
    

    or preferably a typed capability:

    approval.verify(
      plan=effective_plan,
      payload=raw_approval_payload
    ) -> approval_verification
    

    Then:

    sys.assert(
      'approval_verification.valid == true',
      'The human approval evidence does not authorize the current effective deployment plan'
    )
    

    The LLM may select and call the tool, but the resulting variable should have a deterministic origin.

    Choose one invalid-evidence control-flow mechanism

    The current graph contains:

    verify_approval
            ↓
    verified{Evidence valid?}
            ├── yes → deploy
            └── no  → blocked
    

    while verify_approval also contains a failing sys.assert.

    Those are two different invalid-evidence mechanisms.

    If sys.assert fails inside verify_approval, the graph may never reach:

    verified → no → blocked
    

    A cleaner design would use one of these two patterns.

    Pattern A: decision routing

    verify tool
            ↓
    verified{valid?}
            ├── yes → final sys.assert → deploy
            └── no  → blocked
    

    Pattern B: assertion routing

    verify tool
            ↓
    sys.assert
            ├── pass → deploy
            └── assertion_error → blocked
    

    Either is valid. Combining both makes the failure path ambiguous.

    An explicit terminal node should also be added so all of:

    deployed
    cancelled
    blocked
    

    reach a declared End state.

    The deployment capability should be explicit

    The current package declares:

    filesystem
    code
    

    but later says:

    Run agents-cli deploy
    

    For a hard-enforcement Runtime, the Skill needs either:

    shell / sys.run
    

    or, preferably:

    agents_cli.deploy
    

    as a restricted typed capability.

    Another clean design would be for this Skill to end with:

    verified authorization result
    

    and hand off to the existing deployment Skill, rather than directly owning deployment mechanics.

    Either design can work, but the capability boundary should be explicit.

    Verification hardening still worth completing

    The v3 composition is strong, but several fields that are signed should also be checked against Runtime expectations:

    profile
    skill_id
    operation
    plan_schema
    audience
    

    A signature proves those values were not modified.

    It does not prove they are the values this Runtime expected.

    The verifier should therefore return named checks such as:

    {
      "profile_matches": true,
      "skill_id_matches": true,
      "operation_matches": true,
      "plan_schema_matches": true,
      "audience_matches": true
    }

    The included JSON Schemas are also useful, but they should become executable validation, not documentation-only resources.

    Likewise:

    • replay checking should be wired into the actual v3 verification path;
    • nonce consumption should occur only after all static and signature checks pass;
    • production replay state should be durable and atomic;
    • a trusted key_id → public_key resolver should be available when policy requires trusted signing identity.

    An envelope that supplies both its own signature and its own public key proves internal consistency, but not that the key is trusted.

    That remains optional policy, as already agreed, but the distinction should stay explicit.

    Scope naming

    The current implementation is already a strong deployment-approval Skill.

    It does not yet implement the entire original readiness proposal:

    eval threshold enforcement
    IAM role validation
    secrets-policy enforcement
    Python build compatibility
    rollback executability
    observability verification
    post-deploy health validation
    runtime trace emission
    

    That is not a criticism of the current work.

    It may be better engineering to make:

    deployment approval
    

    excellent first, then compose the remaining readiness gates.

    But until those gates exist, I would describe this package as:

    agents-cli deployment approval and plan-binding AISP skill

    rather than implying that the complete deployment-readiness lifecycle is already implemented.

    The next decisive artifact

    The most persuasive next milestone would be one end-to-end SoulBot execution trace:

    1. Load google_agents_cli_deploy_readiness_aisp
    2. Resolve a real Cloud Run or Agent Runtime plan
    3. Stop at WAITING_USER
    4. Human replies with approve {...}
    5. Resume with raw USER_ANSWER
    6. Call approval.verify
    7. Produce structured verification checks
    8. Pass sys.assert
    9. Invoke a dry-run or non-production deployment handoff
    10. Modify region and demonstrate deterministic rejection
    

    Alongside that trace, publish the results of the AISP release checks:

    AISP package validation
    README projection check
    SKILL.md bridge/projection check
    resource-boundary validation
    package hash
    Runtime trace validation
    

    That would close the three most important proof obligations:

    the plan represents the action
    the package conforms to AISP boundaries
    the Runtime actually enforces the contract
    

    Final assessment

    This is outstanding work.

    You have now moved far beyond responding to a speculative Issue.

    You have:

    independently verified a Runtime
    understood the AISP/AISOP responsibility boundary
    implemented real cryptographic evidence
    found a real nondeterminism defect through fixtures
    read the target CLI source
    built a source-grounded resolver
    documented real coverage gaps
    implemented fail-closed supplementation
    constructed an AISP candidate package
    and tested a composed pass/tamper-fail path
    

    That is a rare level of responsiveness and technical seriousness.

    The most impressive part is not the volume of code. It is the consistent refusal to fabricate certainty where the underlying system does not expose a real value:

    no rollback field → mark absent
    no source revision → mark partial or absent
    no IdP → call approver a signed claim
    no durable replay store → call it an in-memory demo
    no SoulBot execution yet → say so directly
    

    That honesty is exactly what security, protocol, and agent-governance work require.

    I would now consider this the first serious external AISP candidate integration grounded in another project's real implementation rather than a synthetic example.

    The remaining issues are closure issues:

    complete the action-binding surface
    make the package self-contained
    make the AISOP bindings unambiguous
    run official conformance checks
    execute it through SoulBot
    

    They do not undermine the architecture you have built.

    They define the path from an excellent external candidate integration to a credible end-to-end reference implementation.

    Thank you again for taking every review point seriously, independently checking the claims, and repeatedly converting discussion into executable, falsifiable evidence. This collaboration has become one of the strongest demonstrations so far of what AISP is meant to enable.
20. babyblueviper1 commented on Aug 15, 2026

    

    babyblueviper1

    on Aug 15, 2026

    More actions

    Thank you — that's a genuinely rare kind of review, and it's earned a genuinely concrete reply rather than a thank-you-and-a-promise.

    Two of the closure issues are fixed, verified, and pushed (commit e17f7c7, docs follow-up 15e17cc):

    Package self-containment. scripts/approval_verifier.py's ../deployment_approval_example.py / ../resolve_agents_cli_plan.py sys.path reach and the evals/vectors symlink are gone — both dependency scripts and all 10 vector fixtures are now real files local to the Skill Folder. Verified by copying the whole folder to an isolated location with zero filesystem access to the outer examples/aisp-deployment-approval/ directory and running scripts/approval_verifier.py --demo there: full resolve → refuse-incomplete → supplement → build → verify(valid=true) → tamper(fails both plan_digest_matches and approved_scope_matches_current_plan) → supplement-override-refused sequence passes with zero outside dependency.

    Control-flow ambiguity. Went with your Pattern A. verify_approval.step2 no longer contains a sys.assert — it just returns the structured verification result. verified{Evidence valid?} is now the sole routing node (no→blocked), with deploy.step1's own sys.assert as the redundant final gate immediately before the irreversible action. Updated the non_negotiable.enforced_by field to name both explicitly instead of pointing at the removed assert.

    Also applied your two doc-level notes while I was in there: README now says "candidate" and reserves "conforming" for after official AISP validation evidence exists, and both README/SKILL.md now describe the scope as "deployment approval and plan-binding," not the full readiness lifecycle — the eval-threshold/IAM/secrets/rollback/observability gates you listed are correctly out of scope for what's built today.

    On the rest — I'm not going to promise a timeline on the full remaining list, because most of it isn't a doc fix, it's a real redesign, and promising it before building it would be exactly the kind of unsupported certainty this thread has been arguing against:

    • Execution-input vs. policy-supplement split + digest binding is the one I think matters most and is next: the resolver should refuse to let an execution input (service account, image identity, port, build args, env vars) enter only the approved plan without also becoming an input to the actual deploy invocation. That's a real architecture change (something closer to your --plan-json sketch, prototyped adapter-side first), not a field reshuffle.
    • Unmapped CLI flags (--update-env-vars, --agent-identity, --port, --build-args, --cluster-name, --no-wait) is mechanical follow-on work once the digest-binding question above is settled — no point mapping more fields into a plan whose binding to the actual executed action isn't solid yet.
    • Immutable image/source-tree digest for production mode and explicit AISOP tool-result binding (sys.run(...) / a typed approval.verify(...) capability instead of implicit) are both real and both smaller than the digest-binding item — plausible next after it.
    • Signed-but-unchecked field verification (profile, skill_id, operation, plan_schema, audience matches, not just signature validity) is a good, contained addition to the verifier — no architectural dependency on the others, could land independently.
    • The SoulBot execution trace is the right final proof and I want to run it, but it's honest to say it's gated on the above landing first — a trace through a Runtime enforcing an approval-plan boundary that doesn't yet bind execution inputs would just demonstrate the gap you already named, not close it.

    So: two real fixes shipped today, one clear next architectural item identified and not yet started, and the rest sequenced behind it rather than promised in parallel. I'll post back here when the execution-input binding lands.
21. babyblueviper1 commented on Aug 15, 2026

    

    babyblueviper1

    on Aug 15, 2026

    More actions

    Shipped the execution-input binding gap -- approval_verifier.py deploy-check (9e489de).

    Since agents-cli can't be modified from here (no native --plan-json), the adapter-level substitute: take the already-approved plan, re-resolve a fresh plan from the live flags/manifest immediately before deploy, and compare every field in EXECUTION_RELEVANT_FIELDS (deployment_target, gcp_project, region, service_name, service_account, resource_sizing, network_exposure, secret_policy_digest, source_revision). Any divergence raises PlanDivergenceError and refuses -- fail closed, same severity as an invalid signature. That's your own named shape: resolve effective plan → approve that plan → deploy consumes the same resolved values, not resolve-approve-then-independently-re-resolve.

    --supplements is split accordingly: policy/evidence fields (eval_evidence, rollback_plan, observability_requirements, environment, python_version) can still fill honestly-absent gaps; execution-input fields are refused outright now, on the theory that if it's execution-relevant it has to come from something that can actually bind to deploy, not a side channel into the approval record alone.

    The negative control is test_deploy_binding.py (10 tests): a service_account drift and a region drift are both asserted to actually flip bound to False, not just checked for a reason field -- same discipline you're holding recompute-kit#13 to, a control that would fail if the compare were a no-op.

    Also closed the smaller named gap: unmapped CLI flags (--update-env-vars, --agent-identity, --port, --build-args, --cluster-name, --no-wait) are now split execution-relevant vs control-flow and printed in the mapping table, instead of silently absent.

    Still open, correctly not attempted here: immutable image/source-tree digest for production mode, the AISOP tool-result binding (sys.run vs an implicit call), signed-but-unchecked field verification (profile/skill_id/operation/plan_schema/audience), and the SoulBot execution trace. Same sequencing as before -- naming them, not promising a date on them yet.
22. asrujana-44

    added

    needs review[Status] The PR/issue is awaiting review from the maintainer[Status] The PR/issue is awaiting review from the maintainer

    on Sep 25, 2026
23. optimization2026 commented on Sep 30, 2026

    

    optimization2026

    on Sep 30, 2026

    Author

    More actions

    Thanks for the concrete follow-through on package self-containment, failure-path routing, and the new deploy-check step. The distinction between execution inputs and policy/evidence supplements is particularly useful.

    The approved-versus-fresh-plan comparison is meaningful progress as an adapter-level pre-deployment drift check. I would keep that description separate from complete execution binding: matching the selected fields at check time does not yet establish that the subsequent deployment consumes exactly those checked inputs.

    For the next bounded increment, the key invariant would be:

    Approval verification and plan matching gate the actual dispatch, and dispatch consumes the same validated execution inputs rather than independently resolving them again.

    For a strict deployment path, execution-affecting inputs that the adapter cannot account for should cause refusal or an explicit unsupported/incomplete result—not be hidden behind an unconditional bound: true. A matched mutable image tag or source label also does not establish immutable artifact identity.

    Could you provide a current full commit link and the exact command for the deployment-binding tests? The short 9e489de reference did not resolve when checked, so a full permalink would make the implementation easier to review reproducibly.

    A small test using a stubbed deployment target would be especially helpful:

    • Invalid approval or an execution-input mismatch results in zero dispatch calls.
    • Matching, valid inputs reach dispatch with the same validated values.
    • An unsupported execution-affecting input cannot silently produce a successful strict-path result.

    No live cloud deployment is needed for that test. It would demonstrate the adapter’s dispatch gate, not claim that the underlying CLI is impossible to invoke through another path.

    One housekeeping clarification: the AISP JSON in the original issue remains an early draft with the conformance corrections identified earlier in this thread. It should not be used as a validated implementation template.

    Thanks also for explicitly separating completed changes from the remaining work. A compact, reproducible adapter follow-up is a sensible next step; any upstream agents-cli PR can then be scoped around a concrete integration need, with final scope and acceptance left to the maintainers.
24. babyblueviper1 commented on Sep 30, 2026

    

    babyblueviper1

    on Sep 30, 2026

    Last edited by babyblueviper1

    More actions

    Thanks. That's a fair correction. The earlier result was an adapter-level pre-deployment drift check, not execution binding. I'll use that wording from here on.

    Full commit: babyblueviper1/invinoveritas@577c107
    (9e489de was a hash from our private working repo. I should have linked the public mirror.)

    Folder: https://github.com/babyblueviper1/invinoveritas/tree/577c1075cbb1eb56ae249759e133dd37a5768f89/examples/aisp-deployment-approval/google_agents_cli_deploy_readiness_aisp

    Test command (stubbed target, no cloud calls):

    git clone https://github.com/babyblueviper1/invinoveritas && cd invinoveritas
    git checkout 577c1075cbb1eb56ae249759e133dd37a5768f89
    pip install coincurve pyyaml
    cd examples/aisp-deployment-approval/google_agents_cli_deploy_readiness_aisp/scripts
    python3 -m unittest test_deploy_binding -v    # 20 tests

    What changed, mapped to your three cases:

    dispatch_deploy(plan, approval, flags, manifest, dispatch_fn) in approval_verifier.py is now the only path that calls the deploy target. It calls dispatch_fn only after verify_payload and check_deploy_binding both pass. It passes the values read from the matched live re-resolve, not the approved plan and not the caller's raw flags, so dispatch uses exactly the inputs that were compared. Any failure raises DispatchRefused before the target is touched.

    TestDispatchGate uses a recording stub as the target:

    • test_invalid_approval_zero_dispatch_calls and test_execution_input_mismatch_zero_dispatch_calls: a forged approval plan_sha256 and a service-account drift each raise, and the stub is called zero times.
    • test_matching_valid_inputs_dispatch_with_same_validated_values: the stub is called exactly once, with values equal field-for-field to the validated execution inputs.
    • test_unsupported_execution_flag_cannot_reach_dispatch: this is a mapped-fields-all-match case with --update-env-vars present. Unmapped execution-affecting flags (--update-env-vars, --port, --agent-identity, --build-args, --cluster-name) now set bound: false through unsupported_execution_flags, so the stub is never called. Control-flow flags such as --no-wait don't affect bound, and test_control_flag_present_does_not_affect_bound covers that.

    Scope, stated plainly: this proves the adapter's own dispatch gate. It doesn't prove agents-cli can't be invoked some other way.

    Artifact identity (your mutable-tag point) is also in this commit. dispatch_deploy(..., require_immutable_artifact=True) refuses unless the matched image is pinned by digest (<repo>@sha256:<64 hex>). A matched :abc123 tag is refused with zero dispatch calls, and so is a from-source deploy with no --image. Outside strict mode, the result still reports artifact.immutable. TestStrictArtifactIdentity covers tag refused, digest dispatched with that exact reference, and from-source refused.

    Still open:

    • Signed-but-unchecked fields (profile, skill_id, operation, plan_schema, audience).

    Noted on the AISP JSON in the original issue: it's an early draft and not a validated template. The README calls this example a candidate, not conforming.
25. optimization2026 commented on Sep 30, 2026

    

    optimization2026

    on Sep 30, 2026

    Author

    More actions

    Thanks for the public permalink and the concrete implementation. The new dispatch_deploy() entrypoint addresses the distinction raised in the previous review: it checks approval and input consistency before calling the deployment target, rather than merely returning a status that a separate caller might ignore.

    The recording-stub tests are particularly useful because they assert zero target calls on refusal and inspect the values actually passed on success. The strict digest-reference option is also a useful addition. Your explicit boundary—this controls the adapter’s dispatch path, not every possible invocation of agents-cli—is the right one.

    The observations below are based on the source and test definitions at 577c1075cbb1eb56ae249759e133dd37a5768f89. They should not be read as an independent rerun of the 20-test suite.

    1. The next priority is connecting the native AISP path to the tested gate

    In the aisp.aisop.json at that same commit, the deploy node still describes:

    deploy-check
        → assert deploy_check.bound
        → separately run agents-cli deploy with the live flags
    

    It does not yet invoke the new dispatch_deploy() path.

    That is the main integration gap: the Python tests exercise the improved gate, while the native skill still describes the earlier check-then-deploy arrangement.

    I would prioritize exposing the tested entrypoint through an explicit runtime/tool binding and making the AISP node use that binding, with deterministic inputs, results, and failure handling. Preserve the real sys.io.confirm step and the unmodified approval payload; the model should not reconstruct signed fields or decide whether verification succeeded.

    The deployment callback should consume only the validated execution inputs. There should be no separate shell-deployment fallback after a refusal.

    A focused integration test should enter through the skill’s actual tool binding—not just call the Python helper directly—and show that refusal produces zero target calls, while a successful case passes the complete expected execution-input set to the target exactly once. Any mocked runtime or human-response component should be identified, so the test does not imply that real human interaction was exercised.

    2. Keep the machine-readable binding separate from its explanation

    The second non_negotiable.enforced_by currently contains:

    verified{Evidence valid?}:no->blocked, and redundantly by deploy.step1:sys.assert immediately before the irreversible action
    

    That explains the intended behavior, but it is not one of the supported AISP binding forms.

    For the current node layout, the precise existing assertion reference is:

    "enforced_by": "deploy.step1:sys.assert"

    Keep the additional topology explanation in the rule text or documentation, and update the binding if the integration changes the step layout. Correcting the reference establishes a valid static binding; it does not by itself establish that the new dispatch gate is being used.

    A pinned upstream validation report would also help identify any remaining package or graph issues before changing the description from “candidate” to “conforming.”

    3. Make the verifier’s remaining assurance boundaries precise

    Reading verify_payload() and the underlying verifier, there are three distinctions worth preserving:

    Audience versus other signed context. The default dispatch path already supplies expected_audience, and verify_approval() compares it. The remaining expected-value checks appear to concern profile, skill_id, operation, and plan_schema. Those expectations should come from trusted verifier configuration, not from the approval being checked. A correctly signed approval for the wrong operation or skill is the useful negative test; changing a field without re-signing would only demonstrate signature verification.

    Replay protection. The underlying verifier accepts an optional replay_guard, but the current dispatch path does not supply one. Signed nonce and max_uses fields therefore do not establish replay prevention on that path. Either connect the guard and test repeated dispatch, or explicitly report replay enforcement as absent and remove “non-replayed” from the dispatch docstring. A durable, shared, atomic store remains a separate production requirement; this iteration need not implement an entire production service.

    Signer authority. Verification currently uses the public key carried in the approval envelope. That is appropriate for a self-contained cryptographic demonstration, but a valid signature under that key does not establish that the signer is authorized to approve deployment. Keep that limitation explicit until a trusted-key or host-identity policy is connected. No particular identity provider needs to be mandated here.

    One small test-coverage note: test_from_source_deploy_refuses_in_strict_mode currently exercises immutable_artifact_identity() rather than the complete dispatch path. Either add a dispatch-level case asserting zero calls and the intended refusal reason, or describe its coverage as helper-level. Digest-reference checking should likewise remain distinct from verifying the bytes fetched and used by a real deployment backend.

    ---

    I would keep the next increment focused on alignment between the native skill, the tested adapter entrypoint, and the claims made about that path—not expand it into the full deployment-readiness lifecycle.

    I also need to correct my earlier mention of an upstream PR: agents-cli currently accepts issue-based feedback but not pull requests. Continuing the adapter work in your repository and bringing back a compact summary of any concrete CLI integration requirements is the appropriate route.

    Thanks again for responding with code, inspectable tests, and clearer scope. Connecting the existing AISP candidate to the gate already implemented would be a meaningful next milestone without requiring a live cloud deployment.
26. babyblueviper1 commented on Sep 30, 2026

    

    babyblueviper1

    on Sep 30, 2026

    More actions

    Thanks. All three points are addressed in babyblueviper1/invinoveritas@81da1ff (skill folder: examples/aisp-deployment-approval/google_agents_cli_deploy_readiness_aisp).

    1. The native AISP path uses the tested gate. The deploy node's only deploy step is now the deploy_dispatch tool: scripts/approval_verifier.py dispatch --plan … --approval … --flags … --manifest … --config verifier_config.json.

    • That command is the only path that runs agents-cli deploy. Exit 0 means it dispatched once; exit 3 means it refused and nothing ran.
    • step3 asserts deploy_result.dispatched == true, and a refusal routes to blocked with no shell fallback. sys.io.confirm and the unmodified approval payload are unchanged.
    • It runs agents-cli deploy once, with argv built from the same flag snapshot the binding check confirmed.

    test_skill_dispatch.py (13 tests) reads that command from aisp.aisop.json and runs it as a subprocess, with $AGENTS_CLI_BIN pointing at a recording stub. It shows:

    • zero calls on a tampered approval, an execution-input drift, a replay, a strict-mode mutable tag, and a from-source drift;
    • exactly one call, with the complete expected argv, on a valid approval.

    The mocked parts are named in the test and the README: the stub target, and harness-built approvals. No sys.io.confirm interaction or AISOP runtime is exercised.

    2. enforced_by is now deploy.step1:sys.assert. The topology explanation moved to the README. The README still says "candidate": I don't have a pinned upstream validation report yet, and I'll attach one before that wording changes.

    3. Assurance boundaries:

    • Domain. profile, skill_id, operation and plan_schema expectations come from verifier_config.json, which is trusted verifier config, not the approval. The negative tests use correctly re-signed approvals for another operation and for another skill, and both are refused with zero calls.
    • Replay. Replay is enforced on the dispatch path with a file-backed, locked, single-host store. The nonce is consumed last, so a failed check does not burn it, and a second dispatch is refused (tested). The docstring no longer says "non-replayed" unconditionally, and an unset store reports replay: "not_enforced". A shared durable store stays a stated production requirement.
    • Signer authority. With no trusted_public_keys, the result reports signer_authority: "not_established". With a key list set, an untrusted but validly signing key is refused (tested).
    • From-source, a finding. A from-source plan can't be approved at all, because the approval builder refuses a plan with no source_revision. So the strict from-source refusal is unreachable through a valid approval. The dispatch-level case that can occur is live flags dropping --image under an image-approved plan, and that is refused with zero calls, strict or not.
    • Artifact identity. The digest check applies to the reference passed. It does not verify the bytes a real backend fetches, and the README says so.

    One CLI integration requirement this surfaced: the adapter has to turn a checked flag snapshot back into argv, because agents-cli deploy only takes flags. If agents-cli deploy accepted the resolved plan as its only execution input (a --plan-json), the gate could bind to what actually executes instead of to a reconstructed command line. That's the single change on the CLI side that would close most of the remaining distance.

    cd scripts && python3 -m unittest test_skill_dispatch test_deploy_binding -v   # 33 tests