Agents in CI/CD

An agent in CI is an agent with no human watching it. It reads the whole checkout, runs the build, and feeds the output back into the model — unattended, on a runner you do not sit in front of. It is the case where a content firewall earns its keep, and also the case where a misconfiguration is least likely to be noticed, because nobody is looking at the terminal.

The good news is that CI is the easy configuration. There is no settings panel and no cached login: the agent reads environment variables, and you control them.

Why put a CI agent behind the firewall

  • Your provider key never sits in the workflow. The runner holds a Vulnetix API key. Your provider key stays in the KMS-encrypted BYOK vault and is decrypted just-in-time by the gateway, on the server side of the request — so it is never a repository secret, never in a workflow log, and never exposed to a pull request from a fork.
  • Every prompt the agent sends is screened by your guardrails — and in CI that is the entire checkout, the build output, and whatever the test suite printed, fed back as tool results.
  • Provider and model policy is enforced centrally. A workflow file cannot silently switch to a model your organisation has not approved: the decision is made at the gateway, on every request, not in a YAML file anyone with write access can edit.

The second and third points are the ones that change in CI. A developer’s laptop is governed by a config file you asked them to install; a runner is governed by a workflow file that any contributor can propose a change to. The gateway is the only place that check cannot be edited away.

The one secret

Store your Vulnetix API key as a repository (or organisation) secret named VULNETIX_API_KEY.

Settings → Secrets and variables → Actions → New repository secret
Name:   VULNETIX_API_KEY
Value:  vlx_...

That is the only credential CI needs. Your OpenAI or Anthropic key does not go here — it goes in the vault, once, from a workstation:

vulnetix ai-firewall key set anthropic --from-env ANTHROPIC_KEY
Tip This is the practical argument for BYOK that people feel immediately: rotating a provider key across twenty repositories used to mean twenty secret updates. Now the provider key lives in one vault, and the repositories hold a Vulnetix key that grants nothing except the right to be screened by your own firewall.

Claude Code in GitHub Actions

anthropics/claude-code-action runs Claude Code on a runner. It speaks the Anthropic Messages API, so it takes the same three environment variables as Claude Code on your laptop — pointed at the gateway’s messages surface.

name: Claude
on:
  issue_comment:
    types: [created]

jobs:
  claude:
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write
    steps:
      - uses: actions/checkout@v4

      - uses: anthropics/claude-code-action@v1
        env:
          ANTHROPIC_BASE_URL: https://guardrails.vulnetix.com/anthropic/${{ vars.VULNETIX_ORG_UUID }}
          ANTHROPIC_AUTH_TOKEN: ${{ secrets.VULNETIX_API_KEY }}
          ANTHROPIC_API_KEY: ""

The three rules are exactly the ones from the Claude Code page:

  1. No /v1 on ANTHROPIC_BASE_URL. The Anthropic client appends it itself.
  2. ANTHROPIC_AUTH_TOKEN is your Vulnetix API key, not an Anthropic key.
  3. ANTHROPIC_API_KEY must be an explicit empty string. A stale value here conflicts with the auth token and produces confusing authentication failures.

The organisation UUID is not a secret — it is a URL component. A repository variable (vars.VULNETIX_ORG_UUID) keeps it out of the workflow body without pretending it needs protecting.

Warning Guard the trigger, not just the traffic. Firewalling the model calls does not stop an untrusted pull request from causing them. Gate agent workflows on trusted actors — an if: condition on the comment author’s association, or a manual label — exactly as you would for any workflow that can write to the repository.

Other agents in CI

Anything that runs on a laptop runs on a runner, and the wiring is the same environment-variable exercise. For an OpenAI-compatible agent — Aider is the common one in CI — set the base URL in the config and the credential in the environment:

      - name: Run the agent
        env:
          OPENAI_API_KEY: ${{ secrets.VULNETIX_API_KEY }}
        run: |
          aider --openai-api-base "https://guardrails.vulnetix.com/openai/${{ vars.VULNETIX_ORG_UUID }}/v1" \
                --model gpt-4o-mini \
                --yes --message "..."

Note the shapes, which differ by protocol and trip people up when they move a working laptop config into CI: the OpenAI base URL ends in /v1; the Anthropic one stops at the organisation UUID. See base URLs.

Gate the build on it

The point of CI is that it fails loudly. vulnetix ai-firewall status --strict exits non-zero when a check fires, which turns “is this agent actually behind the firewall?” into a build step rather than a thing somebody remembers to look at.

      - name: Firewall posture
        env:
          VULNETIX_API_KEY: ${{ secrets.VULNETIX_API_KEY }}
        run: vulnetix ai-firewall status --strict

The checks it can fail on are the same ones you see locally: bypasses_firewall (a client configured to talk to a provider directly), provider_denied, provider_key_missing, model_denied, model_not_allowed, wire_unsupported, key_env_unset and guardrail_pattern_invalid.

Run it before the agent step, not after. A provider_key_missing or model_not_allowed caught by the gate costs you a fast red build; caught by the agent itself, it costs you a half-finished pull request and a confusing log.

This is also the check that catches a workflow drifting off the gateway: someone “fixing” a failing agent by putting a raw provider key in a secret and pointing the base URL back at the vendor is precisely the bypasses_firewall case, and the gate sees it.

Limitations

  • The gate checks configuration, not traffic. status --strict tells you the runner is wired correctly. It cannot tell you that no step in the job called a provider directly with a key of its own — a curl to api.openai.com with a hardcoded key is invisible to it, as it is to any client-side check.
  • The arguments the model generates for a tool call are not scanned. Prompt content and tool results are.
  • Anyone who can edit the workflow can unwire it. The firewall is a control, not a cage. Protect the workflow file with required reviews, the same way you protect anything else in CI that holds a secret.

Common errors

401 — invalid Vulnetix API key

{
  "type": "error",
  "error": {
    "type": "authentication_error",
    "code": "invalid_api_key",
    "message": "invalid Vulnetix API key"
  }
}

In CI this is nearly always one of two things. The two-keys confusion: the secret named VULNETIX_API_KEY actually contains an Anthropic or OpenAI key, left over from before the firewall. Or the secret is simply not visible to the job — a secret defined on an environment the job did not name, or a workflow triggered from a fork, where secrets are withheld by design. An unset secret expands to an empty string, and an empty credential is an invalid one.

Fix:

  1. Confirm the secret’s value is your Vulnetix key (vlx_…), not a provider key. Your provider key belongs in the vault: vulnetix ai-firewall key set anthropic --from-env ANTHROPIC_KEY.
  2. Confirm the job can see the secret — check the environment: on the job, and whether the trigger is a fork pull request.
  3. Add vulnetix ai-firewall status --strict as an earlier step; its key_env_unset check fails the build before the agent gets a confusing 401.

403 — provider_key_missing

{
  "type": "error",
  "error": {
    "type": "permission_error",
    "code": "provider_key_missing",
    "message": "no Anthropic API key configured for your organisation: add one with 'vulnetix ai-firewall key set anthropic' or at https://www.vulnetix.com/vdb-ai-firewall#byok"
  }
}

The runner authenticated fine, but your organisation has no key in the vault for the provider in the base URL, so the gateway has nothing to forward the request with. This is a common first-run error in CI, because the workflow was written before anyone stored the key.

Fix:

  1. From a workstation: export ANTHROPIC_KEY=sk-ant-...
  2. vulnetix ai-firewall key set anthropic --from-env ANTHROPIC_KEY
  3. vulnetix ai-firewall status — the provider’s row should now show a stored key.
  4. Re-run the job. Nothing in the workflow needs to change.

403 — model_not_allowed

{
  "type": "error",
  "error": {
    "type": "permission_error",
    "code": "model_not_allowed",
    "message": "model claude-opus-4-1 is not in the organisation allowlist"
  }
}

The model the action or CLI requested is not on your organisation’s allowlist. (A model that is explicitly denied returns model_denied instead.) In CI this often surfaces after an action’s default model changes underneath you on a new release — the workflow did not change, but what it asks for did.

Fix:

  1. Run vulnetix ai-firewall get to see which models your policy permits.
  2. Pin the model explicitly in the workflow rather than relying on the action’s default.
  3. If the model should be allowed, an administrator adds it to the policy.

404 — unsupported_api

{
  "type": "error",
  "error": {
    "type": "not_found_error",
    "code": "unsupported_api",
    "message": "provider openai does not support the Anthropic Messages API (POST /v1/messages). Use POST /v1/chat/completions for openai, or point your Anthropic client at a provider that serves the Messages API: https://guardrails.vulnetix.com/anthropic/{orgUuid} (note: no /v1 suffix — the Anthropic SDK appends it)"
  }
}

The base URL is wrong. For an Anthropic client, it must name the anthropic provider and stop at the organisation UUID — appending /v1 gives the client’s own /v1 something to collide with. For an OpenAI client, the /v1 is required, and the provider must be one that serves the chat surface (or openai for responses).

Fix:

  1. Print the base URL your job is actually using — an interpolation that silently produced an empty vars.VULNETIX_ORG_UUID is the usual culprit, and it yields a URL missing its organisation segment.
  2. Anthropic: https://guardrails.vulnetix.com/anthropic/{orgUuid} — no /v1.
  3. OpenAI: https://guardrails.vulnetix.com/{provider}/{orgUuid}/v1 — with /v1.
  4. Confirm the repository variable is defined at the level the workflow reads it from.