AI

AI lets your App call a large language model from your handler code — a chat reply, a summarization step, anything that needs an LLM in the loop. Your App always calls sdk.ai.*.

AI is pure BYOK (Bring Your Own Key). Anyknown does not proxy LLM traffic, does not front any platform credit, and does not pay for these calls on your behalf. You supply your own provider API key as a secret, and the provider bills you directly. This is different from every other Capability with a “default provider” — there is no Anyknown-managed fallback key.

Five providers are supported, each wrapped through the Vercel AI SDK so the call shape is identical regardless of which one you pick:

Provider key	AI_PROVIDER value	Secret name
Anthropic	"anthropic" (default)	ANTHROPIC_API_KEY
OpenAI	"openai"	OPENAI_API_KEY
Google	"google"	GOOGLE_GENERATIVE_AI_API_KEY
xAI	"xai"	XAI_API_KEY
Fireworks	"fireworks"	FIREWORKS_API_KEY

Set up your provider key

AI is available once your provider’s secret is set. Use set_secret (or anyknown apps secrets set) to store the key — you never paste it into code; the SDK reads it through the platform on each call. Anyknown has not smoke-verified end-to-end delivery against every provider/key combination — the SDK surface on this page is final and stable to build against, but the last hop to your own provider account is on you to verify.

anyknown apps secrets set ANTHROPIC_API_KEY <your-key-here>

The key is encrypted at rest and injected into the App Worker at call time via the credential resolver — it never appears in plaintext env bindings, in your source, or in logs. The platform-managed ak.* secret namespace is walled off from BYOK keys; you cannot borrow a platform secret, and the platform cannot read yours.

If a call needs a key that is not set, the SDK throws AiError with kind: "secret-missing" (status 412).

Chat — non-streaming

chat sends one request and resolves with the full response text.

chat(input: AiChatInput): Promise<AiChatResult>

type AiChatInput = {
  messages: { role: "user" | "assistant"; content: string }[]
  system?: string
  model?: string          // optional — uses the provider default if omitted
  maxOutputTokens?: number
  consent?: "granted" | "denied" | "pending" // default "pending" — see consent section below
}

type AiChatResult = {
  text: string
  usage: { inputTokens?: number; outputTokens?: number; totalTokens?: number; cachedInputTokens?: number }
}

const sdk = c.get("sdk")
const result = await sdk.ai.chat({
  messages: [{ role: "user", content: userMessage }],
  system: "You are a helpful assistant.",
})
return c.json({ reply: result.text })

Streaming — SSE passthrough

stream returns a web ReadableStream<Uint8Array> you can hand straight to a Response for server-sent-events passthrough to the browser. Same input shape as chat.

stream(input: AiChatInput): Promise<ReadableStream<Uint8Array>>

const sdk = c.get("sdk")
const stream = await sdk.ai.stream({
  messages: [{ role: "user", content: userMessage }],
  system: "You are a helpful assistant.",
})
return new Response(stream, {
  headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache" },
})

Choosing a model

Each provider has a fixed, versioned catalog of selectable models. Omit model to use the provider’s current default; passing an id outside the catalog (or outside your App’s allowlist, if one is configured) throws AiError with kind: "unknown-model".

Provider	Default model
anthropic	claude-sonnet-4-5
openai	gpt-5
google	gemini-2.5-flash
xai	grok-4
fireworks	accounts/fireworks/models/llama-v3p3-70b-instruct

An App’s Customer can additionally set an AI_MAX_INPUT_CHARS ceiling (default 24 000 characters, hard-capped at 200 000 by the platform) and an AI_MODEL_ALLOWLIST — a subset of the global catalog — without triggering a full App regeneration. A request whose total input (messages + system prompt) exceeds the configured limit throws AiError with kind: "input-too-long" (status 413) before any provider call is made.

Consent and I/O logging

The optional consent field controls whether prompt and response content is retained for Caretaker analysis and debugging.

consent value	Content stored	Metadata stored
"granted"	Yes — PII-scrubbed before storage	Yes
"denied"	No	Yes (token counts, model, time)
"pending" (default when omitted)	No	Yes

This defaults to fail-closed — the opposite of Analytics, which defaults to granted. AI content is Visitor prompt/response text, not just event identifiers, so no content is retained unless you explicitly pass consent: "granted". Scrubbing runs in your App Worker before anything leaves it, with a secondary check on the platform side as a backstop. Retained content lives in a short-retention log table, separate from the token-count metering used for observability.

Per-App request quota

A per-App request quota throttles aggregate AI throughput (default 60 requests/minute). It is fail-open — a quota-store fault lets requests through rather than blocking your App.

This is a per-App quota, not a per-Visitor limit

If your App exposes an AI endpoint to Visitors without authentication, a single abusive Visitor can exhaust your BYOK provider budget and run up real provider costs — the per-App quota does not stop them. Put your own Identity check or a bot-defense mechanism in front of any public-facing AI endpoint.

Handling errors

Every call can throw AiError. It carries a structured kind and an HTTP status.

type AiErrorKind =
  | "validation" // 400 — malformed input, or a reserved tool-calling field was set
  | "not-available" // — capability not available in this context
  | "secret-missing" // 412 — provider key not set
  | "unknown-provider" // 400 — AI_PROVIDER binding holds an unrecognized value
  | "unknown-model" // 400 — model not in the catalog or App allowlist
  | "input-too-long" // 413 — input exceeds the configured character limit
  | "rate-limited" // 429 — per-App request quota exceeded
  | "auth" // 502 — provider rejected the key
  | "circuit-open" // 503 — provider circuit breaker is open
  | "provider-error" // 502 — provider returned a 5xx, or the stream failed

import { AiError } from "@anyknown/app-server"

const sdk = c.get("sdk")
try {
  const result = await sdk.ai.chat({ messages: [{ role: "user", content: userMessage }] })
  return c.json({ reply: result.text })
} catch (err) {
  if (err instanceof AiError) {
    return new Response(err.message, { status: err.status })
  }
  throw err
}

Things to know

• Pure BYOK — no platform credit. Every call is billed by the provider directly against the key you supplied. Anyknown never pays for or proxies your App’s LLM usage.

• Chat and stream only in v1. sdk.ai.chat() and sdk.ai.stream() cover conversational completions. Agent loops and Customer-defined tool-calling are not available yet — the input type reserves the seam (tools, toolChoice, stopWhen, maxSteps), but v1 rejects any of those fields with a validation error at the boundary. Don’t build against tool-calling today.

• No embeddings, image generation, or transcription. sdk.ai is chat and stream only.

• Consent defaults to no content retained. Pass consent: "granted" explicitly if you want Caretaker to see Visitor prompt/response content; the default keeps only token counts and timing.

• Token counts are metered, not billed. Usage is recorded for observability. There is no $ cost conversion, and no Anyknown-side billing tied to AI usage — your provider bills you directly.

• No per-Visitor rate limit yet. The request quota is per-App aggregate. If your AI endpoint is public, front it with your own auth or bot defense.