Rolling out a rule

The failure mode of an LLM firewall is not a missed detection. It is a rule that fires on legitimate work, blocks a developer mid-task, and gets the whole thing switched off by lunchtime.

The flag action exists to prevent that. Use the ladder.

The ladder

Week 1 — `flag`

The rule runs. Matches are recorded. Nothing is blocked and nothing is rewritten.

vulnetix ai-firewall policy guardrail no-connection-strings \
  --rule-type blocked_pattern \
  --action flag \
  --pattern '(?i)(postgres|mysql)://[^\s]+:[^\s]+@' \
  --priority 10 --enable

You now have signal with zero risk. Let it run against real traffic — a week is usually enough, longer if your usage is bursty.

Note

flag records that a rule matched and which rule it was. It never records what matched — the content that triggered it is not stored anywhere, ever. You learn the rule fires 40 times a week; you do not get to read the 40 prompts.

That is a deliberate trade. It means adopting a guardrail never creates a new repository of sensitive text. It also means tuning is done by reasoning about the pattern, not by reading the traffic.

To see the matches, your organisation needs inference logging enabled — the rules that fire are recorded on each request’s log row (as metadata, never content).

Week 2 — decide

Look at the volume:

Fires never. Either your organisation genuinely does not do this, or your pattern is wrong. Check it compiles: vulnetix ai-firewall status.
Fires occasionally. This is the shape you want. Promote it.
Fires constantly. You have a false-positive problem, or a much bigger real problem than you thought. Do not promote it to block yet — go and find out which. See tuning false positives.

Week 3 — promote

vulnetix ai-firewall policy guardrail no-connection-strings \
  --uuid <uuid> --action block

Get the UUID from vulnetix ai-firewall get. Updates are partial — only the flags you pass are changed, so this touches the action and nothing else.

Which rung to stop on

flag is not only a staging state. It is a legitimate destination.

Action	Use it when
`flag`	You want visibility, not enforcement. The behaviour is undesirable but not dangerous, or you cannot yet write a pattern precise enough to enforce on.
`redact`	The model does not need the data to do its job. PII, mostly. The user gets a useful answer and the data never leaves.
`block`	The data must not leave, full stop, and the user needs to know. Secrets, credentials, keys.

Most organisations end up with a mix, and that is correct. A policy that is all block is one that has not been tuned.

Tell your developers

A block is loud — the error names the rule that fired, so a developer who hits one learns the policy exists in the moment it matters. That is good design and you should lean on it.

A redaction is silent. The developer gets a normal answer about redacted text and no indication why. If you are going to redact, tell people you redact, or you will generate a stream of “the model is being stupid” tickets that are actually your firewall working as designed.

Do the whole thing in a file

The ladder is much easier to run as policy as code. Change one line, open a pull request, and the diff is the change:

guardrails:
  - name: no-connection-strings
    ruleType: blocked_pattern
    action: flag        # ← week 3: change to `block`
    pattern: '(?i)(postgres|mysql)://[^\s]+:[^\s]+@'
    priority: 10
    enabled: true

vulnetix ai-firewall apply --dry-run   # see exactly what would change
vulnetix ai-firewall apply

Your security policy gets reviewed like any other change, and you have a history of why each rule is where it is.

Tuning false positives — what to do when a rule is too eager.
Policy as code — reviewed, versioned, applied from CI.
Opt-in logging — required to see what your flag rules are catching.