AI

SyntaxKit ships a complete org-scoped streaming chat at /dashboard/ai-chat on top of the Vercel AI SDK and the Vercel AI Gateway. Everything routes through one adapter, gateway(modelId), so swapping providers is a string change rather than a refactor. The @syntaxkit/ui/components/ai-elements/* package gives you the same composer, transcript, attachments, reasoning, sources, model picker, and mic button to drop into any new AI feature.

We pick the AI Gateway over per-provider SDKs because it removes the "N keys for N providers" problem: one Vercel key, dozens of model ids, one billing surface, one place to switch when a new model lands.

How A Chat Turn Flows

The unusual bit is the transport. The kit pairs @ai-sdk/react's useChat with a custom transport.sendMessages that calls the oRPC streaming procedure client.chat.send and pipes the result through eventIteratorToUnproxiedDataStream from @orpc/client. The server returns streamToEventIterator(result.toUIMessageStream({ sendReasoning: true, sendSources: true })) so reasoning and sources arrive as first-class UI message parts. Persistence happens in onFinish: the user message is saved before the stream, and the assistant row stores both content (the answer text) and parts (reasoning, text, and source-url citations) plus an AiUsageEvent once tokens stop flowing. Reload reconstructs the thinking block and citations from those parts.

The diagram source lives at apps/docs/diagrams/ai-chat-flow.mmd. Rerun pnpm --filter @syntaxkit/docs diagrams:build after editing it to refresh both SVG variants.

Package Layout

What's Wired In

The AI SDK And Gateway

Two design choices to call out before wiring:

The model registry is CHAT_MODEL_REGISTRY in packages/shared/src/ai/chat-models.ts, the one file you edit to change which models the app offers. Each entry declares the gateway model id, a display name, the provider (name plus the models.dev slug used for the picker logo and grouping), an enabled flag, its capabilities (reasoning shows the effort control and maps it onto provider options for OpenAI, Anthropic, Google, xAI/SpaceXAI, Moonshot, and Z.AI; vision shows attach and lets the server accept image parts), and optional limits.maxOutputTokens. The same file sets defaultModelId (exported as CHAT_DEFAULT_MODEL_ID) and webSearchModelId (exported as CHAT_WEB_SEARCH_MODEL_ID). The registry is validated at import time, so a typo in the default id or a duplicate entry fails the boot and the test suite with a precise message.

The server chat layout calls getChatModelPickerOptions(CHAT_MODEL_REGISTRY) and passes the result through the chat shell to the picker: enabled models only, in registry order, without the web-search model or any server-only knobs. The server is authoritative: resolveChatModel in @syntaxkit/core/chat accepts a requested model only if it is an enabled, selectable registry entry, for both chat.send and chat.regenerate. Unknown ids, disabled ids, and a direct request for the web-search model return BAD_REQUEST before any quota is reserved. Adding a model is therefore one entry in the registry; retiring one is enabled: false.

Env vars. The chat path needs exactly one AI-specific env: AI_GATEWAY_API_KEY (recognized by the AI SDK's gateway() adapter). It is documented in apps/web/.env.example, modeled as the optional aiChat setup capability, and reported by pnpm setup:doctor.

Browse provider and model metadata at models.dev. To offer another model, add its AI Gateway ID to packages/shared/src/ai/chat-models.ts; chat requests only accept enabled entries in that registry.

The Streaming Procedure

chat.send is a single-purpose oRPC procedure (POST /rpc/chat/send) that flows through five gates before it ever opens a model connection. Listed in order:

After the gates pass, the user Message is persisted, then:

const result = streamText({
  model: gateway(modelId),
  instructions: CHAT_TOOLS_INSTRUCTIONS,
  messages: modelMessages,
  tools: createChatTools(ctx), // omitted when webSearch is on
  stopWhen: stepCountIs(CHAT_TOOLS_STOP_STEPS),
  onError: async () => {
    // Refund the reservation row so a failed turn does not consume the cap.
  },
  onFinish: async ({ text }) => {
    // Persist the assistant Message and (on the first turn)
    // generateText({ model: gateway(CHAT_TITLE_MODEL_ID) }) to set
    // Chat.title and flip Chat.titleGenerated. The AiUsageEvent row has
    // already been written by reserveAiUsageEvent before streaming began.
  },
});

return streamToEventIterator(
  result.toUIMessageStream({
    sendReasoning: true,
    sendSources: true,
  })
);

chat.regenerate mirrors this: same gates, same streamText shape, but onFinish deletes the trailing assistant turn(s) before inserting the replacement and writes an AiUsageEvent with kind: "chat_regenerate" instead of "chat_send".

The Client: useChat With An oRPC Transport

The kit uses @ai-sdk/react's useChat over an oRPC streaming procedure rather than a plain fetch endpoint. The bridge is one helper:

import { useChat } from "@ai-sdk/react";
import { eventIteratorToUnproxiedDataStream } from "@orpc/client";
import { client } from "@/lib/orpc";

const { messages, sendMessage, status, stop } = useChat({
  id: chatId,
  messages: seedMessages,
  transport: {
    async sendMessages(options) {
      const latestMessage = options.messages[options.messages.length - 1];
      return eventIteratorToUnproxiedDataStream(
        await client.chat.send(
          {
            chatId: options.chatId,
            messages: buildSendPayloadMessages(latestMessage),
            model: webSearchRef.current ? undefined : modelRef.current,
            webSearch: webSearchRef.current,
          },
          { signal: options.abortSignal }
        )
      );
    },
    reconnectToStream() {
      throw new Error("Unsupported");
    },
  },
  onFinish: () => {
    // Invalidate the sidebar list and dashboard stats.
  },
});

A few patterns worth knowing:

AI Elements: Building Blocks

Reusable presentation primitives. Group them by what they do, not what file they live in. All export from @syntaxkit/ui/components/ai-elements/<file>.

Transcript Primitives

Composer Primitives

Streaming-Aware Primitives

Billing And Limits

Two limit surfaces gate AI usage. Per-plan features control what a user is allowed to do; per-request limits control what fits in a single payload.

Per-plan features

The cap is enforced by counting AiUsageEvent rows (kind: "chat_send" or "chat_regenerate") in the current monthly quota window. See Billing for how plans and entitlements are configured, and Storage: How An Upload Flows for the attachment pipeline that feeds chat images.

Per-request limits

All defined in packages/shared/src/schemas/chat.ts and enforced by Zod on the server.

History is trimmed newest-first inside buildModelContextMessages, so an active conversation always keeps its tail and the trim shows up only on long threads.

Abuse Protection

chat.send and chat.regenerate are surfaces in the shared abuse policy. Each request is keyed by userId and organizationId; both characteristics must be present, otherwise the surface fails closed.

Tuning happens in packages/shared/src/abuse.ts: each surface declares its window and limit per characteristic, so you can lower the per-user cap without changing the per-org cap.

Adding A New AI Feature

Reuse the same handler shape for a new feature (summarize a doc, generate alt text, draft an email, anything). The pattern is the streaming chat in miniature.

export const summarizeInputSchema = z.object({
  text: z.string().min(1).max(50_000),
  style: z.enum(["bullet", "tldr", "executive"]).default("tldr"),
});

import { CHAT_DEFAULT_MODEL_ID } from "@syntaxkit/shared/ai/server";

export const summarize = authorized
  .use(withActiveOrganization)
  .route({ path: "/summarize", method: "POST" })
  .input(summarizeInputSchema)
  .handler(async ({ context, input }) => {
    const billing = await getBillingState(context.organization.id);
    // Atomically reserve a slot in the monthly cap before spending
    // gateway tokens. Refund the reservation if the model errors.
    const usage = await reserveAiUsageEvent(billing, context.organization.id, {
      kind: "chat_send",
      chatId: null,
      createdByUserId: context.user.id,
    });
    try {
      const result = streamText({
        model: gateway(CHAT_DEFAULT_MODEL_ID),
        system: "Summarize the input in the requested style.",
        prompt: input.text,
        onError: () =>
          prisma.aiUsageEvent
            .delete({ where: { id: usage.id } })
            .catch(() => {}),
      });
      return streamToEventIterator(result.toUIMessageStream());
    } catch (error) {
      await prisma.aiUsageEvent
        .delete({ where: { id: usage.id } })
        .catch(() => {});
      throw error;
    }
  });

Where To Go Next