Testing

SyntaxKit ships a four-tier test setup: Vitest for unit, integration, and live Stripe tests, and Playwright for end-to-end browser tests. Every layer runs independently, so you can drive any subset locally while CI runs them all on each push. Per-package coverage gates (90% lines / 85% branches) keep the unit tier honest.

The four layers

Unit, integration, live Stripe, and E2E: what each touches and how to run it.

Test environment

The deterministic .env.test escapes and why each one is enabled.

CI pipeline

How the jobs run on every PR, and where to grab failure traces.

Add a test

Drop in a unit, integration, or E2E spec the right way.

The Test Pyramid

Unit Tests

Co-located *.test.ts(x) files run via per-package Vitest configs: jsdom for component tests, node for backend packages. Each package config (apps/web/vitest.config.ts plus the seven packages/*/vitest.config.ts) declares its own coverage gate, and CI fails the build if any of them slips. Every gate uses the same threshold object:

coverage: {
  thresholds: {
    lines: 90,
    statements: 90,
    functions: 90,
    branches: 85,
  },
},

Worker Budget

Turborepo runs package tasks in parallel and each Vitest instance spawns its own fork pool, so without a cap the two fan-outs multiply: ten packages × (cores − 1) forks each. On an 18-core machine that is ~120 processes and ~9 GB of memory for a suite whose wall time is bounded by the largest package either way. The kit budgets both layers from one place:

Together that is about 75% of logical cores at peak: a 4-core laptop runs 3 forks in total, an 8-core runs 6, and the 4-vCPU GitHub runner runs 3 instead of 30. Keep the two values in step if you change either. For a single run, VITEST_MAX_WORKERS=<n> overrides the per-package pool (it is on the task's passThroughEnv allowlist so Turborepo's strict env mode does not drop it) and pnpm test:run --concurrency=<n> overrides the package count.

Watch mode (pnpm test) is exempt from the package cap: it starts one persistent Vitest watcher per package, and Turborepo refuses to run more persistent tasks than its concurrency allows, so the root script lifts the cap for that path. Idle watchers cost nothing until a file changes, and each one still uses the 25% pool. To watch a single package, use pnpm --filter @syntaxkit/web test.

Integration Tests

Real Postgres, real Prisma, real aws-sdk calls against a Moto-mocked S3 bucket. The *.integration.test.ts suffix lives outside the unit-test include glob, so pnpm test:run skips them; you opt in with pnpm test:integration. The harness in packages/api/test/integration.ts exposes createIntegrationHarness() with ensureStorageBucket, cleanup, and getStoredObject helpers:

import { createIntegrationHarness } from "../../test/integration";

const harness = createIntegrationHarness();

beforeEach(async () => {
  await harness.ensureStorageBucket();
  await harness.cleanup();
});

afterEach(async () => {
  await harness.cleanup();
});

Live Stripe Tests

The opt-in tier. Test-mode Stripe behavior occasionally drifts from what unit-test mocks assume, so the kit ships a small live suite that catches that drift early. Files use *.live.test.ts or *.live.integration.test.ts, gated by RUN_STRIPE_LIVE=1 so they never run during a normal pnpm test:run. CI runs them on workflow_dispatch only, via .github/workflows/stripe-live.yml. Locally:

RUN_STRIPE_LIVE=1 pnpm test:stripe

Polar Tests

The Polar adapter (packages/payments/src/polar/) is covered by two tiers:

• Unit tests (*.test.ts, run by pnpm test:run) mock @polar-sh/sdk and sign webhook fixtures with the standardwebhooks library, so verification, both secret encodings, event parsing, correlation, status normalization (unpaid to canceled), the first-subscription trial rule, price checks, and reconcile outcomes run without credentials. Renewals, trial expiry, and dunning are exercised here with fixture payloads, because the Polar sandbox has no test clocks to fast-forward a subscription.
• A live sandbox test (packages/payments/src/polar/checkout-portal.live.test.ts) creates a customer, a checkout, and a portal session against the real sandbox API. It is opt-in and refuses to run unless POLAR_SERVER=sandbox:

RUN_POLAR_LIVE=1 POLAR_SERVER=sandbox POLAR_ACCESS_TOKEN=polar_oat_... \
  POLAR_PRODUCT_ID_PRO_MONTHLY=<uuid> pnpm --filter @syntaxkit/payments test:live

There is no root test:polar script and no scheduled Polar workflow; the sandbox test runs through the payments package's test:live task. apps/web/.env.test.example ships sandbox placeholder values for the Polar variables (a placeholder token and nil-pattern product UUIDs) that the unit tier and the compose E2E job use; the live test skips itself when it sees them.

Creem Tests

Creem unit tests mock the SDK and sign creem-signature fixtures. They run in pnpm test:run with billing off. The live sandbox file packages/payments/src/creem/checkout-portal.live.test.ts is skipped unless RUN_CREEM_LIVE=1, CREEM_SERVER=test, a non-placeholder CREEM_API_KEY, and CREEM_PRODUCT_ID_PRO_MONTHLY are set. CI never requires those keys. Buyers who leave billing off do not run it.

End-to-End Tests

Playwright with Chromium. Specs live in apps/web/e2e/ and cover every critical user flow: marketing, auth, signup, onboarding, dashboard, organization-settings, personal-settings, billing, two-factor, admin. Playwright starts the web server itself via the webServer config in apps/web/playwright.config.ts:

webServer: {
  command: `npx dotenvx run --overload -f .env.test -- next start --port ${TEST_PORT}`,
  env: { NODE_ENV: "test" },
  url: TEST_BASE_URL,
  reuseExistingServer: !process.env.CI,
  timeout: 30_000,
},

The deterministic escapes (DISABLE_ABUSE_PROTECTION, DISABLE_CAPTCHA_FOR_TESTS, empty UPSTASH_*) are loaded from apps/web/.env.test by dotenvx. See The Test Environment for what each one does. The auth specs in apps/web/e2e/auth.spec.ts exercise the same flows documented on Authentication, so changes to login, signup, password reset, or 2FA all re-verify against a real browser.

Parallelization And Worker Fixtures

Playwright runs four workers locally and two in GitHub Actions, and authenticated specs avoid global state by giving each worker its own deterministic user. Three files stay in sync to make this work:

The fixture in apps/web/e2e/helpers/fixtures.ts caches the sign-in once per worker: the first spec signs the user in and writes storageState to a file, and every later spec in that worker reuses the cached cookie state:

workerStorageState: [
  async ({ browser }, apply) => {
    const id = test.info().parallelIndex;
    const fileName = path.resolve(
      test.info().project.outputDir,
      `.auth/${id}.json`
    );

    if (fs.existsSync(fileName)) {
      await apply(fileName);
      return;
    }

    const user = getWorkerUser(id);
    const page = await browser.newPage({ storageState: undefined });
    // ...sign in, then:
    await page.context().storageState({ path: fileName });
    await apply(fileName);
  },
  { scope: "worker", timeout: 120_000 },
],

Signup specs create new users instead of reusing seeded ones, so uniqueSignupEmail() returns a pid + Date.now() + counter string and concurrent workers never collide on the same email.

The Test Environment

apps/web/.env.test (copied from apps/web/.env.test.example) defines the deterministic test environment. Several escapes are enabled deliberately so tests don't fight production-grade abuse and rate-limit policies. Each is hard-blocked at boot in production.

Mocking Patterns

The canonical recipe lives in packages/api/src/router/storage.integration.test.ts: top-of-file vi.mock("@syntaxkit/shared", ...) with vi.importActual to selectively keep real exports, then vi.fn() for the boundaries you want to control:

vi.mock("@syntaxkit/shared", async () => {
  const actual = await vi.importActual("@syntaxkit/shared");

  return {
    ...actual,
    enforceAbusePolicy: vi.fn(async () => ({
      ok: true,
      reason: "allowed",
      results: {},
    })),
  };
});

Mirror this for any test that mocks a cross-package boundary. Use vi.hoisted() when the mock factory needs data the mocked module reads at load time (rare: it surfaces in middleware and config tests). Pure-function tests need no mock; just call the function and assert.

CI Pipeline

.github/workflows/ci.yml runs the full suite on every pull request and on release tags. It does not rerun the suite on the merge to main: that second copy of the same suite did not block the merge. Pushes to main run only the secret scan and a cache warm-up (below). Jobs use GitHub-hosted ubuntu-latest unless you set the Actions variables CI_RUNNER and CI_RUNNER_ARM (for example ubicloud-standard-2 / ubicloud-standard-2-arm). A changes job detects which areas of the repo a change touches; every other job installs the workspace through the shared .github/actions/setup composite action (lockfile-keyed caches for node_modules, the Prisma client, and package build output):

• Checks & Tests runs pnpm lint, pnpm check-types, pnpm db:validate, the build-arg drift check, and the Agent Skills mirror check. It builds the docs site when docs inputs changed, and runs the unit tier when product code changed. Pull requests run pnpm test:run. Release tags run pnpm test:coverage so the per-package coverage gates are enforced on what ships.
• Setup Smoke runs only when first-run files change (setup:doctor, .env.example, Prisma schema, next.config.ts). It writes a core-mode .env with the three required values and proves pnpm setup:doctor and a production next build succeed.
• Integration Tests boots Postgres 17 and Moto S3 containers, applies migrations and seeds, then runs pnpm test:integration.
• E2E Tests boots Postgres 17, builds the web app once with the test-only Next config (output in .next-test), installs Chromium, applies migrations and seeds, and runs Playwright. Green runs keep a compact timing summary; failures and retried tests upload the HTML report and traces.
• Secret scan runs gitleaks on pull requests, tags, and every push to main, so a direct push that skipped review is still scanned.
• Warm caches runs on pushes to main that change a cache key input (lockfile, Prisma schema, composite package sources). It runs the setup action and installs Chromium so those caches are saved in main's cache scope. Pull requests can only restore caches from their own branch and from main, so without it every new PR would install cold. No tests run in this job.
• Docker and Release run on release tags only. Docker builds and smokes the multi-arch images by digest in parallel with the tests; Release moves :<version> and :latest only after CI Gate passed. A tag with a failing test never publishes an image tag. To retry a failed publish, re-run the failed jobs of that CI run.
• CI Gate is the single required status check. It passes when every job passed or was skipped by the path filter and fails on any real failure.

The Stripe live suite runs on a separate workflow (.github/workflows/stripe-live.yml) because it depends on real third-party credentials and shouldn't gate every PR. The full suite is workflow_dispatch only. A catalog price-drift check runs on pushes to main that touch billing files and weekly, because a price edited in the provider dashboard changes no file. It checks every provider (Stripe, Polar, Creem) whose secrets the repository has and skips the rest. The Docker image publish (docker.yml) does not run on ordinary pushes to main; dependency changes on main only warm the Docker deps layer cache. Pull requests that change the Dockerfile, deploy config, workspace manifests, lockfile, Next config, or Prisma schema build and smoke the web and migrator images without pushing. Version tags publish through ci.yml as described above; a manual run publishes the current branch without moving :latest, and a manual run on a v* tag republishes only if that tag's CI Gate passed. The Compose stack check (compose-e2e.yml) runs on pushes to main that change the Dockerfile or compose config, not on every product change.

Adding A Test

Where To Go Next