LlamaIndex

LlamaIndex’s OpenAI LLM wraps the OpenAI SDK, so the firewall is two constructor arguments away: api_base and api_key. Your indexes, retrievers, query engines, and agents are all downstream of the LLM object and do not change.

Install

pip install llama-index-llms-openai

Configure

Have the CLI print the snippet with your organisation UUID already in it:

vulnetix ai-firewall snippet --lang python --sdk llamaindex --provider openai

Or by hand:

import os
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI

org = os.environ["VULNETIX_ORG_UUID"]

llm = OpenAI(
    api_base=f"https://guardrails.vulnetix.com/openai/{org}/v1",
    api_key=os.environ["VULNETIX_API_KEY"],  # Vulnetix key, not an OpenAI key
    model="gpt-4o-mini",
)

Settings.llm = llm  # every query engine and agent now goes through the firewall

print(llm.complete("Hello"))

Note the argument is api_base, not base_url — LlamaIndex’s own naming. Point the slug at any provider (groq, mistral, deepseek, openrouter) and use that provider’s model string; the gateway forwards model verbatim.

Streaming

Unchanged.

for chunk in llm.stream_complete("Hello"):
    print(chunk.delta, end="", flush=True)

Handling a guardrail block

LlamaIndex does not wrap SDK exceptions, so a guardrail refusal surfaces as the underlying OpenAI error: an HTTP 403 raised as openai.PermissionDeniedError. It comes out of complete, chat, and out of any query engine or agent step that calls the model.

import openai

try:
    response = query_engine.query(question)
except openai.PermissionDeniedError as e:          # HTTP 403
    err = e.response.json().get("error", {})
    code = err.get("code")                          # request_blocked, model_denied, ...
    rule = err.get("blocked_by")                    # the rule that stopped it
    if code == "request_blocked":
        raise RuntimeError(f"AI firewall blocked this query: rule {rule!r}") from e
    raise

`code`	Meaning
`request_blocked`	A content guardrail matched. `blocked_by` names the rule; `violations` lists every rule that matched.
`provider_denied`	Policy does not allow this provider.
`model_denied` / `model_not_allowed`	Policy does not allow this model.
`provider_key_missing`	No provider key in the vault for this provider.

Note Redaction is silent — nothing is raised. A pii_redact rule rewrites matched spans to the literal [REDACTED] and lets the request through. This is the case worth internalising for RAG: the prompt a query engine builds contains your retrieved nodes, so a document with PII in it is redacted in the prompt, and the model answers as though that text had never been there. No exception, no warning — just an answer with a hole in it.

Tool / function calling

Function tools and agent workflows pass through untouched. Guardrails inspect the request LlamaIndex sends, and on an agent’s second turn that request carries the tool results from the first — so a tool that pulls a secret into the conversation produces a request the firewall can block or redact.

Gotchas

api_key is your Vulnetix key. A provider key here is the most common 401.
LlamaIndex reads OPENAI_API_BASE and OPENAI_API_KEY from the environment, so vulnetix ai-firewall install shell can route an existing script with no code change. Passing api_base explicitly keeps the gateway in the code path regardless of the shell.
api_base must include the /v1 suffix.
Embeddings do not go through the firewall. The gateway serves chat completions, the Responses API, and Anthropic’s Messages API — there is no embeddings surface, so pointing OpenAIEmbedding at it returns 404. Leave your embedding model talking to its provider directly; the firewall’s job is the prompts, and an embedding request carries the same text your retrieval pipeline already had.