Schema Patterns

Forge’s Convex backend has 87+ tables spanning workflow execution, agent management, governance, chat, budgets, analytics, and more. This page documents the patterns and conventions that keep the schema organized, performant, and safe to evolve.

Schema Organization

Every table lives in its own file inside convex/schema/. The root convex/schema.ts imports each table definition and composes them into a single defineSchema() call using the spread operator. Why one file per table:

Each table definition is independently readable and reviewable
Git diffs stay scoped to the table that changed
Enum constants and TypeScript types are co-located with the table they belong to
No 3,000-line monolith schema file

The Composition Pattern

Each schema file exports an object with a single key (the table name) whose value is a defineTable(...) call. The root schema spreads them all together:

// convex/schema/budgets.ts
import { defineTable } from "convex/server";
import { v } from "convex/values";
import { literals } from "./helpers";

export const BUDGET_SCOPES = ["global", "gateway", "agent"] as const;
export type BudgetScope = (typeof BUDGET_SCOPES)[number];

export const BUDGET_PERIODS = ["daily", "weekly", "monthly"] as const;
export type BudgetPeriod = (typeof BUDGET_PERIODS)[number];

export const budgetsTable = {
  budgets: defineTable({
    scope: literals(BUDGET_SCOPES),
    scopeId: v.optional(v.string()),
    period: literals(BUDGET_PERIODS),
    amount: v.number(),
    enabled: v.boolean(),
    spentCents: v.optional(v.number()),
    periodStart: v.optional(v.number()),
    lastResetAt: v.optional(v.number()),
    createdAt: v.number(),
    updatedAt: v.number(),
  })
    .index("by_scope", ["scope", "scopeId"])
    .index("by_scope_period", ["scope", "scopeId", "period"])
    .index("by_enabled", ["enabled"]),
};

// convex/schema.ts (root — abbreviated)
import { defineSchema } from "convex/server";
import { budgetsTable } from "./schema/budgets";
import { factoriesTable } from "./schema/factories";
import { workflowRunsTable } from "./schema/workflow_runs";
// ... 80+ more imports

export default defineSchema({
  // Budget Management
  ...budgetsTable,
  ...budgetHierarchyTable,
  ...budgetTransactionsTable,
  ...budgetAlertsTable,

  // Workflow Execution
  ...workflowsTable,
  ...workflowRunsTable,
  ...stepResultsTable,
  // ...
});

The root file groups imports by domain (Gateway & Infrastructure, Workflow Execution, Budget Management, etc.) with comments acting as section headers. This makes it easy to find where a new table should be registered.

Barrel Exports for Enum Constants

convex/schema/index.ts re-exports every enum constant array and its derived TypeScript type from every schema file. This lets consuming code import enums from a single path:

// Consumer code can import enums cleanly
import { STEP_STATUSES, BUDGET_SCOPES } from "../schema";
import type { StepStatus, BudgetScope } from "../schema";

Table Definition Patterns

Enum Fields with `literals()`

The convex/schema/helpers.ts file provides a literals() helper that converts a TypeScript as const tuple into a Convex v.union(v.literal(...)) validator:

// convex/schema/helpers.ts
export function literals<const T extends readonly [string, string, ...string[]]>(
  values: T,
): Validator<T[number]> {
  const [first, second, ...rest] = values.map((val) => v.literal(val));
  return v.union(first!, second!, ...rest) as unknown as Validator<T[number]>;
}

Every enum field follows the same three-line pattern:

export const FACTORY_STATUSES = ["PROVISIONING", "ACTIVE", "PAUSED", "ARCHIVED"] as const;
export type FactoryStatus = (typeof FACTORY_STATUSES)[number];

// In the table definition:
status: literals(FACTORY_STATUSES),

This gives you:

A runtime-accessible array of valid values (for dropdowns, validation, etc.)
A TypeScript union type for compile-time safety
A Convex validator that enforces the constraint at the database level

Document References with `v.id()`

References to other tables use v.id("tableName"). This creates a typed, validated foreign key:

// Hard reference — this document MUST exist in the referenced table
runId: v.id("workflowRuns"),

// Optional reference — may or may not be set
gatewayId: v.optional(v.id("gatewayRegistry")),
budgetHierarchyId: v.optional(v.id("budgetHierarchy")),
parentRunId: v.optional(v.id("workflowRuns")),   // self-referential
logStorageId: v.optional(v.id("_storage")),        // Convex system table

v.id() validates that the string is a valid document ID for that table, but does not enforce referential integrity at the database level. Deleting a referenced document will not cascade or fail — your application code must handle dangling references.

Optional Fields with `v.optional()`

Wrap any validator with v.optional() when the field may not exist on all documents. Common reasons:

Additive evolution: New fields added to existing tables (old documents lack them)
Discriminated unions: Fields that only apply to one variant (e.g., goal only exists on delegated runs, not prescriptive ones)
Progressive enrichment: Fields populated asynchronously after initial creation (e.g., completedAt, token counts)

// From workflowRuns — discriminated by runType
goal: v.optional(v.string()),              // only for delegated runs
definition: v.optional(v.string()),        // only for prescriptive runs
guardrails: v.optional(v.object({...})),   // only for delegated runs

// Progressive enrichment
completedAt: v.optional(v.number()),
totalTokens: v.optional(v.number()),
totalCost: v.optional(v.number()),

Nested Objects

Use v.object() for structured sub-documents. Keep nesting shallow (1-2 levels) to maintain queryability:

// Pricing snapshot embedded in token events
pricingSnapshot: v.object({
  input: v.number(),
  output: v.number(),
}),

// Channel references on a factory
channelIds: v.optional(v.object({
  requests: v.optional(v.id("channels")),
  support: v.optional(v.id("channels")),
  general: v.optional(v.id("channels")),
  engineering: v.optional(v.id("channels")),
})),

Dynamic Fields with `v.any()`

Some fields intentionally accept arbitrary shapes. These are rare and always documented:

// DYNAMIC: user-defined runtime arguments, shape varies per workflow
args: v.optional(v.any()),

// DYNAMIC: agent output is unpredictable by design
outputs: v.optional(v.any()),

// DYNAMIC: factory-type-specific configuration, shape varies per factory type
config: v.optional(v.any()),

Avoid v.any() unless the data shape is genuinely unpredictable. If you can enumerate the possible shapes, use v.union() with v.object() variants instead. When v.any() is unavoidable, add a comment explaining why.

Timestamps

Tables use numeric timestamps (v.number()) representing epoch milliseconds. Most tables include at least createdAt and updatedAt:

createdAt: v.number(),
updatedAt: v.number(),

There is no auto-timestamping. Your mutation code is responsible for setting these values via Date.now().

Index Strategy

Indexes are declared by chaining .index() calls on the defineTable() result. Forge follows consistent naming and design conventions.

Naming Convention

Pattern	Example	Usage
`by_fieldName`	`by_status`	Single-field lookup
`by_field1_field2`	`by_scope_period`	Composite lookup
`by_field1_time`	`by_agent_timestamp`	Range queries scoped to an entity

Single-Field Indexes

For simple filtering and lookup:

.index("by_status", ["status"])
.index("by_type", ["factoryType"])
.index("by_enabled", ["enabled"])
.index("by_workflow", ["workflowId"])

Composite Indexes

For scoped queries where you filter on one field and range-scan on another. The equality fields come first, followed by the range field:

// "Find all token events for this agent, ordered by time"
.index("by_agent", ["agentId", "timestamp"])

// "Find all token events for this model, ordered by time"
.index("by_model", ["model", "timestamp"])

// "Find budgets for this scope and period"
.index("by_scope_period", ["scope", "scopeId", "period"])

// "Find all runs of this type with this status"
.index("by_runType", ["runType", "status"])

Multi-Dimension Scoped Indexes

For hierarchical data with multiple scoping levels:

// Channels scoped by type, entity type, and entity ID
.index("by_scope", ["scope", "scopeEntityType", "scopeEntityId"])

// Factories scoped by owner type and owner ID
.index("by_owner", ["ownerType", "ownerId"])

// Workflow runs by status and completion time (for cleanup/retention queries)
.index("by_status_completedAt", ["status", "completedAt"])

When to Add an Index

Add an index when...

You query a field with .withIndex("by_field", q => q.eq("field", value))
You need to range-scan within a scoped entity (e.g., “all events for run X after time T”)
A dashboard page lists records filtered by status, type, or owner
You run retention/cleanup jobs that scan by timestamp within a status

Skip an index when...

The table is small (< 1,000 documents) and full scans are acceptable
You only ever look up documents by _id
The field has very low cardinality and the table is large (index won’t help much)
You can reuse an existing composite index (Convex supports prefix matching on composite indexes)

Convex composite indexes support prefix queries. An index on ["scope", "scopeId", "period"] also efficiently serves queries that only filter on ["scope"] or ["scope", "scopeId"]. You do not need separate single-field indexes for prefix subsets of a composite index.

Validator Patterns

Convex Validators vs. Zod Schemas

Forge uses two validation systems for different purposes:

System	Where	Purpose
Convex validators (`v.*`)	`convex/schema/`	Database-level schema enforcement
Zod schemas (`z.*`)	`packages/shared/`	Input validation, API contracts, `.lobsterX` parsing

They are complementary. Convex validators define what the database accepts. Zod schemas validate external input before it reaches the database.

The `.lobsterX` Zod Schema Example

The .lobsterX workflow format is validated with Zod in packages/shared/src/lobsterx-schema.ts. It demonstrates several patterns: Regex-enforced naming conventions:

const snakeCasePattern = /^[a-z][a-z0-9_]*$/;
const kebabCasePattern = /^[a-z][a-z0-9-]*$/;

id: z.string().regex(snakeCasePattern, "Step id must be snake_case"),

Typed sub-schemas composed into a parent:

export const outputFieldSchema = z.object({
  type: z.enum(["string", "number", "integer", "boolean", "object", "array"]),
  description: z.string().optional(),
  required: z.boolean().default(true),
  // ...
});

export const lobsterXStepSchema = z.object({
  id: z.string().regex(snakeCasePattern, "Step id must be snake_case"),
  command: z.string().optional(),
  role: z.string().optional(),
  depends_on: z.array(z.string()).optional(),
  outputs: z.record(z.string(), outputFieldSchema).optional(),
  // ...
});

Cross-field validation with .superRefine():

export const outputFieldSchema = z.object({
  type: z.enum(["string", "number", "integer", "boolean", "object", "array"]),
  minimum: z.number().optional(),
  maximum: z.number().optional(),
  // ...
}).superRefine((field, ctx) => {
  if (field.minimum != null && field.maximum != null && field.minimum > field.maximum) {
    ctx.addIssue({
      code: z.ZodIssueCode.custom,
      message: `minimum (${field.minimum}) must be <= maximum (${field.maximum})`,
      path: ["minimum"],
    });
  }
});

Migration Awareness

Convex handles schema changes differently than traditional SQL databases. Understanding these differences is critical before modifying tables.

Additive Changes (Safe)

These changes deploy without issues:

Adding a new table
Adding a new optional field (v.optional(...)) to an existing table
Adding a new index to an existing table
Adding a new value to an enum array (though the literals() validator must be updated)

Breaking Changes (Require Planning)

These changes can fail deployment if existing data doesn’t conform:

Adding a required field (non-optional) to a table that has existing documents
Removing a field that existing documents still have
Changing a field’s type (e.g., v.string() to v.number())
Removing an enum value that existing documents use
Renaming a table (Convex sees this as dropping one table and creating another)

Safe Migration Strategy

For breaking changes, use the widen-migrate-narrow pattern:

Widen

Make the new field optional alongside the old one. Deploy.

Migrate

Run a migration to backfill the new field on all existing documents.

Narrow

Once all documents have the new field, make it required and remove the old field. Deploy.

Never make a required field optional or remove a field in a single deployment if existing documents use it. Convex validates all existing documents against the new schema at deploy time.

Convex Conventions

System Fields

Every Convex document automatically has:

_id — unique document ID (typed as Id<"tableName">)
_creationTime — auto-set epoch timestamp (milliseconds)

You do not declare these in your schema. They are always available on query results.

The `_generated/` Directory

convex/_generated/ contains auto-generated TypeScript types, the api object, and the dataModel type. Never edit files in this directory. They are regenerated on every npx convex dev or npx convex deploy.

Document ID Types

Use v.id("tableName") in schemas and Id<"tableName"> in TypeScript code:

// In schema
runId: v.id("workflowRuns"),

// In function code
import type { Id } from "./_generated/dataModel";

function getSteps(runId: Id<"workflowRuns">) { ... }

File Naming

Convex module files must use underscores, not hyphens. Convex uses the file path as the function’s API path, and hyphens are not valid in Convex identifiers:

convex/schema/workflow_runs.ts     // correct
convex/schema/workflow-runs.ts     // WRONG — will not work

Node.js API Restriction

Standard Convex functions (queries, mutations) run in a V8 isolate without Node.js APIs. If you need fetch, crypto, or other Node.js built-ins, use an action file with the "use node" directive. Never import Node.js APIs in convex/lib/ or schema files.

Adding a Table End-to-End

Follow these steps when adding a new table to the schema.

Create the schema file

Create a new file in convex/schema/ using underscore_case naming:

// convex/schema/review_comments.ts
import { defineTable } from "convex/server";
import { v } from "convex/values";
import { literals } from "./helpers";

export const COMMENT_STATUSES = ["active", "resolved", "deleted"] as const;
export type CommentStatus = (typeof COMMENT_STATUSES)[number];

export const reviewCommentsTable = {
  reviewComments: defineTable({
    runId: v.id("workflowRuns"),
    stepId: v.string(),
    authorId: v.string(),
    body: v.string(),
    status: literals(COMMENT_STATUSES),
    resolvedAt: v.optional(v.number()),
    createdAt: v.number(),
    updatedAt: v.number(),
  })
    .index("by_run", ["runId"])
    .index("by_run_step", ["runId", "stepId"])
    .index("by_author", ["authorId", "createdAt"]),
};

Import the table in convex/schema.ts and spread it into the appropriate domain section:

import { reviewCommentsTable } from "./schema/review_comments";

export default defineSchema({
  // ... existing tables ...

  // Step Execution & Results
  ...stepResultsTable,
  ...reviewCommentsTable,   // add here
  // ...
});

Export enums in the barrel

Add type and constant exports to convex/schema/index.ts:

export type { CommentStatus } from "./review_comments";
export { COMMENT_STATUSES } from "./review_comments";

Create functions

Create a Convex module for queries and mutations against the new table. Use requirePermission() on all public mutations:

// convex/review_comments.ts
import { v } from "convex/values";
import { mutation, query } from "./_generated/server";
import { requirePermission } from "./lib/auth";

export const list = query({
  args: { runId: v.id("workflowRuns") },
  handler: async (ctx, args) => {
    return ctx.db
      .query("reviewComments")
      .withIndex("by_run", (q) => q.eq("runId", args.runId))
      .collect();
  },
});

Run Convex dev

Start (or restart) the Convex dev server to generate types and validate the schema:

pnpm convex:up:detached

The _generated/ directory will update with types for your new table.

Checklist

Before shipping a new table, verify:

File uses underscore_case naming (review_comments.ts, not review-comments.ts)
Enum constants use as const with derived TypeScript types
v.optional() wraps fields that may not exist on all documents
Indexes cover your primary query patterns
Table is registered in convex/schema.ts under the correct domain section
Enum types are re-exported in convex/schema/index.ts
No v.any() without a comment explaining why
Public mutations use requirePermission(), not just requireAuth()

Start Here

Agents

Workflows

Trust Fabric

Observability

Operations

Factories & Chat

Architecture

Execution Flows

Engineering

API Reference

SDK & MCP

​Schema Organization

​The Composition Pattern

​Barrel Exports for Enum Constants

​Table Definition Patterns

​Enum Fields with literals()

​Document References with v.id()

​Optional Fields with v.optional()

​Nested Objects

​Dynamic Fields with v.any()

​Timestamps

​Index Strategy

​Naming Convention

​Single-Field Indexes

​Composite Indexes

​Multi-Dimension Scoped Indexes

​When to Add an Index

​Validator Patterns

​Convex Validators vs. Zod Schemas

​The .lobsterX Zod Schema Example

​Migration Awareness

​Additive Changes (Safe)

​Breaking Changes (Require Planning)

​Safe Migration Strategy

​Convex Conventions

​System Fields

​The _generated/ Directory

​Document ID Types

​File Naming

​Node.js API Restriction

​Adding a Table End-to-End

​Checklist

Schema Organization

The Composition Pattern

Barrel Exports for Enum Constants

Table Definition Patterns

Enum Fields with `literals()`

Document References with `v.id()`

Optional Fields with `v.optional()`

Nested Objects

Dynamic Fields with `v.any()`

Timestamps

Index Strategy

Naming Convention

Single-Field Indexes

Composite Indexes

Multi-Dimension Scoped Indexes

When to Add an Index

Validator Patterns

Convex Validators vs. Zod Schemas

The `.lobsterX` Zod Schema Example

Migration Awareness

Additive Changes (Safe)

Breaking Changes (Require Planning)

Safe Migration Strategy

Convex Conventions

System Fields

The `_generated/` Directory

Document ID Types

File Naming

Node.js API Restriction

Adding a Table End-to-End

Checklist