Documentation

Quick start#

Xalerate Gateway implements the OpenAI /v1/chat/completions format. If you already use the OpenAI SDK or any OpenAI-compatible client, point it at https://api.xalerate.ai and you're done.

curl https://api.xalerate.ai/v1/chat/completions \
  -H "Authorization: Bearer $BO_API_KEY" \
  -H "X-User-Id: alice@acme.com" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      { "role": "user", "content": "Hello from Xalerate Gateway" }
    ],
    "stream": true
  }'

Two headers are required on every request: Authorization: Bearer <api_key> and X-User-Id: <email_or_id>. The user ID is how BO attributes requests to end-users in your audit trail — pass the email of whoever is making the request on your side.

SDK compatibility#

Xalerate Gateway is OpenAI API-compatible, so the official OpenAI Python and Node.js SDKs work as true drop-in replacements. Point base_url at BO, pass the BO API key + X-User-Id (or X-Actor-Token in strict mode), and existing OpenAI-SDK code runs unchanged.

Both stream modes are supported. Streaming is the right choice for interactive UIs; non-streaming is simpler for batch and one-shot processing. The standard client.chat.completions.create() call works for both:

from openai import OpenAI

client = OpenAI(
    api_key="$BO_API_KEY",
    base_url="https://api.xalerate.ai/v1",
    default_headers={"X-User-Id": "alice@acme.com"},
)

# Streaming (interactive UIs)
stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this email: ..."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

BO-specific fields via extra_body#

The OpenAI SDKs only know about the OpenAI field set. Pass BO-specific fields like routing_preferences and pii_policy through extra_body — the wire request is byte-identical to a curl call with the same body.

# BO-specific fields (routing_preferences, pii_policy) via extra_body
response = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "..."}],
    stream=False,
    extra_body={
        "routing_preferences": {
            "compliance": "eu_cloud",
        },
        "pii_policy": {
            "action": "pseudonymize",
            "sensitivity": "medium",
            "rehydrate_response": True,
        },
    },
)

Error handling#

Provider errors return real HTTP 4xx/5xx with a structured JSON body, so the SDK's built-in exceptions work as written. Pre-stream errors (auth, access denied, validation) come back as proper HTTP status codes — your withFallback, retry decorators, and circuit-breakers behave correctly. Mid-stream errors (failures after the first chunk lands) still appear as SSE error frames inside a 200 response — protocol-level constraint once the response is committed. Your SSE parser should check for an error key on any received chunk.

from openai import APIError, RateLimitError, APIStatusError
import time

try:
    response = client.chat.completions.create(...)
except RateLimitError as e:
    # 429 — body carries retry_after_seconds + retry_after_source
    body = e.response.json()
    wait = body["error"].get("retry_after_seconds", 5)
    time.sleep(wait)
    # ...retry
except APIStatusError as e:
    # 4xx — caller-fix needed. Hint is in the body.
    body = e.response.json()
    code = body["error"]["code"]
    if code == "bedrock_access_denied":
        print(body["error"]["hint"])
        # "The requested model is not enabled in your AWS Bedrock account..."
except APIError as e:
    # 5xx — transient. Retry with backoff.
    raise

Node.js#

Same contract — configure with baseURL + defaultHeaders:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "$BO_API_KEY",
  baseURL: "https://api.xalerate.ai/v1",
  defaultHeaders: { "X-User-Id": "alice@acme.com" },
});

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Hello" }],
  stream: false,
});
console.log(response.choices[0].message.content);

What's not supported#

• client.completions.create() — legacy completions API. Use chat.completions.create() instead.
• client.images.*, client.audio.* — multimodal endpoints (image generation, TTS, STT) are catalog-modeled but not yet routable through BO.
• client.files.*, client.batches.*, client.fine_tuning.*, client.moderations.create() — provider-specific features outside BO's scope. Use inline file content blocks instead of files; pii_policy covers most moderation use cases.
• client.embeddings.create() — fully supported via /v1/embeddings.
• seed, response_format, logprobs — passed through to the upstream provider; behavior is provider-dependent.

Anthropic SDK#

The Anthropic SDK targets api.anthropic.com directly and uses the native messages shape. BO does not implement that surface — only OpenAI-compatible. Use the OpenAI SDK pointed at BO for Claude models too (model: "claude-sonnet", model: "claude-opus-4-7", etc.); model selection routes through BO's catalog.

Capturing the request ID#

Every response includes an X-Request-Id header (format: bo_req_<hex>). Capture it client-side and paste into the audit dashboard's "Look up by Request ID" search to cross-reference any individual request — including failure cases.

Model selection#

The model field accepts three kinds of values:

• A specific BO model name — claude-sonnet, gpt-5, gpt-5-mini, gpt-5-nano, mistral-large, magistral-medium (Mistral reasoning, unrestricted tier), gemini-flash, evroc-kimi-k2-6 / berget-kimi-k3 (Kimi open weights, EU-hosted on Evroc / Berget), grok-4-3, grok-3-mini (xAI, unrestricted tier only), and so on. Routes directly to that model.
• auto — BO picks the highest-scoring available model for your project's territorial tier, based on real-time health data and cost optimization.
• A compliance shorthand — eu-strict, eu-cloud, eu-sweden. BO picks the best model qualifying for that tier. Optional size suffix: eu-strict:small.

Territorial tiers#

Every project has a territorial tier that controls which models (and which physical regions) are allowed to serve your requests. Territorial tier is enforced — customer-asserted compliance with a full audit trail.

You can tighten the tier per request (never loosen it) via routing_preferences.compliance:

curl https://api.xalerate.ai/v1/chat/completions \
  -H "Authorization: Bearer $BO_API_KEY" \
  -H "X-User-Id: alice@acme.com" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [...],
    "routing_preferences": {
      "compliance": "eu_strict"
    }
  }'

Model access & key controls#

Within a project's territorial tier, you control which models the project's keys can call, and how a single key authenticates. Both are owner/admin controls in the dashboard, and every change is written to the audit trail.

Auto-approve eligible models

By default a project keeps a curated allowed_models list — keys can only call models on it. Turn on Model access in project settings (default off) and keys can call any model eligible for the project's tier, with new catalog models becoming callable automatically — no list edit.

Each API key can override the project with its Model access control:

An explicit key setting always wins over the project default. Use Force off for a locked-down integration key and Force on for a trusted internal key that should track the full eligible catalog.

BYOB / key-is-actor mode

Trusted-mode projects normally require a per-request X-User-Id header so each call is attributed to an end-user; a bare-Bearer request with no actor fails closed (400 missing_user_id). That is right for an app serving many users, but wrong for a single autonomous agent where the API key itself is the actor.

Turn on BYOB on a key (default off) and a bare-Bearer trusted-mode request from that key is attributed to a synthetic key-is-actor principal (byob:<key_id>) in the audit log instead of returning 400. X-User-Id, when present, still takes precedence — BYOB only fires when no actor header is sent, and only for keys granted the capability. A normal key with no actor still fails closed.

Document input (PDF)#

Send PDFs directly through /v1/chat/completions on any compliance tier. BO accepts three content-block shapes and normalizes them at entry: Anthropic's type: "document", OpenAI's type: "input_file", and the generic type: "file" block. Use whichever your SDK emits.

Per-tier routing: on unrestricted, eu_cloud, and eu_sweden, the PDF is forwarded to the model natively (Anthropic document blocks for unrestricted; Bedrock Converse document blocks for the Bedrock-backed tiers) so the model sees the PDF with layout intact. Your API contract is unchanged — one request in, one response out (streaming or non-streaming, your choice).

Sending a document to a model that doesn't accept documents (for example Gemini models, gpt-o3 and embedding models) returns 400 model_does_not_support_documents with a list of document-capable alternatives for your tier. A document is never silently dropped.

Embeddings#

Xalerate Gateway exposes an OpenAI-compatible POST /v1/embeddings endpoint — a drop-in for vector database and RAG pipelines. Swap the OpenAI embeddings base URL for BO and you get the same governance as chat: territorial routing, per-request audit, and PII protection. Same base URL and auth as chat (Authorization: Bearer bo_live_…, plus X-Actor-Token: bo_actor_… in strict mode).

curl https://api.xalerate.ai/v1/embeddings \
  -H "Authorization: Bearer $BO_API_KEY" \
  -H "X-User-Id: alice@acme.com" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "eu-cloud",
    "input": ["The quick brown fox", "jumps over the lazy dog"]
  }'

The response is OpenAI-compatible — one data row per input, in request order, plus the resolved model and usage:

{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.0021, -0.0145, ...] },
    { "object": "embedding", "index": 1, "embedding": [0.0098, 0.0042, ...] }
  ],
  "model": "mistral-embed",
  "usage": { "prompt_tokens": 12, "total_tokens": 12 }
}

Request parameters#

• model — a concrete embedding model (mistral-embed, mistral-codestral-embed, titan-embed-v2-sweden) or a tier alias (eu-strict, eu-sweden, auto). Tier-aware routing works the same as chat.
• input — a string or an array of strings to embed.
• dimensions (optional) — if supplied, it must match the resolved model's native output dimension, or the request is rejected with 400.

Embeddings are billed per input token (rows have no output cost), and requests land in the same audit trail as chat.

Tier-aware routing#

The tier alias (or your project's territorial tier) selects an embedding model that qualifies for that tier:

PII protection#

When enabled, Xalerate Gateway routes prompts through a Presidio-based PII gateway before any provider call. Detected entities are pseudonymized (or redacted) server-side; the provider sees only the scrubbed version. Token mappings live in a session-scoped vault and are purged after the request completes.

Six languages supported: English, French, German, Italian, Swedish, Spanish. Custom recognizers for Swedish personnummer and IBAN.

curl https://api.xalerate.ai/v1/chat/completions \
  -H "Authorization: Bearer $BO_API_KEY" \
  -H "X-User-Id: alice@acme.com" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet",
    "messages": [
      {
        "role": "user",
        "content": "My name is Karl Karlsson. What's 2+2?"
      }
    ],
    "pii_policy": {
      "enabled": true,
      "sensitivity": "medium",
      "action": "pseudonymize"
    }
  }'

The audit log records pii_policy_received, pii_processed, pii_entities_detected, and pii_redactions_applied on every request that carries a pii_policy field — so you can prove the feature ran by querying /v1/data/audit.

Audit trail#

Requests produce an audit record — success, failure, rate-limit rejection, PII fail-closed, compliance mismatch, and everything in between. The record is written to a durable best-effort outbox before the provider call and advances through a documented state machine; a sweeper closes rows a mid-stream crash left in a non-terminal state.

Query the audit log programmatically via GET /v1/data/audit, filter by request_id (exact, the X-Request-Id of a response), actor_id, model, or status, and get back the full metadata: provider, resolved model, effective tier, cost, tokens, data region, and pricing provenance (what price was in effect at request time).

Advanced features#

• Embeddings — POST /v1/embeddings. Same tier-aware routing as chat completions. Mistral's direct embedding models (mistral-embed, mistral-codestral-embed) are unrestricted-only.
• MCP server — Model Context Protocol JSON-RPC 2.0 at POST /v1/mcp. Compatible with Claude Desktop, Cursor, and any MCP client. Tools: search_audit_logs, list_traces, get_model_health, and more.
• Traces — governed operations for agentic AI. Create a trace with a budget, depth limit, and allowed model list; LLM calls within the trace are gated by the operation contract and logged as a span. See POST /v1/traces.
• Data API — seven read-only endpoints at /v1/data/* for SIEM integration, custom dashboards, and automated reporting. API-key auth only (no actor token required).
• Billing API — GET /v1/billing/balance, /usage, and /reconciliations for programmatic access to spend data, scoped to the calling project.

Pricing#

Xalerate Gateway uses a prepaid balance model. Top up your balance via the dashboard (EUR / USD / SEK); each request atomically deducts its cost from your balance, with full per-request transparency in the audit log. Monthly reconciliation against the actual provider invoice.

Bring your own provider API keys (BYOK) and BO acts purely as the governance + audit layer. You can mix BYOK and Gateway-managed keys per project, and every request is tagged with its key source so billing is unambiguous.

Plans:

• Free — 1,000 requests/month, unrestricted tier only
• Trial — 30 days of Pro access (100,000 requests/month, all tiers). Card required at signup so the trial flips to Pro automatically when it ends.
• Pro — €79 / $79 / 899 kr per month, 100,000 requests/month, all tiers including the EU tiers (eu_strict, eu_cloud, eu_sweden)
• Enterprise — custom pricing, unlimited requests, negotiable convenience fee, SLA, dedicated support

Subscription tiers sit alongside the prepaid balance — Pro unlocks higher request quotas and the EU tiers. Enterprise contracts are negotiated directly.

Xalerate Gateway is operated by Xalerate AB, Stockholm. The Gateway runs on Railway, a US company, in its EU region (europe-west4, the Netherlands).