Test a Data Connector

Overview

A data connector extension can pass its own tests and still fail in Engine. The build doesn't typecheck, and Engine doesn't check the declarations your build returns until a data source (the connector configured in a workspace) is created or reintrospected. It checks them again whenever it loads the stored schema, so an Engine upgrade can disable a declaration that passed before; see Engine Upgrades Can Disable Connector Collections.

A lenient test fixture hides bugs too. It can answer a wrong request to the external data source (the API, service, or database the connector calls) with the right items.

The ladder below runs from cheap to expensive. Each rung catches bugs the rungs before it can't. The snippets come from the Stripe connector in Build a Stripe Data Connector and the full Art Institute of Chicago connector from Map an External Data Source, which the Data Connector Quickstart trims. A file's first excerpt starts at the top of the file, and later excerpts of the same file continue it; // … or # … marks the lines an excerpt leaves out.

Reach rung 5 early, before you write query code. A declaration Engine refuses is cheaper to fix before planning code depends on it.

The example projects have a package.json script for lint, typecheck and unit tests, the build, the artifact smoke and the Engine smoke. Installing, breaking checks on purpose and trying Studio have no script:

Choose a Fixture

Test against an external data source you control, with data you know. Never test against production.

For every fixture:

• Namespace what you create. The Stripe fixture uses an email domain (@stripe-fixture.monospace.test) and custom product ids (prod_msfx_*). The Engine smoke also creates and deletes products named prod_docs_smoke_<timestamp>_*, and creates prices, with ids Stripe assigns, on the fixture product prod_msfx_locked. Writes touch only those items. Paging checks read the account's customers without filtering them to the fixture, and compare with Stripe's own answer. The limit 150 check needs more than 100 customers in the account, which the seed doesn't create.
• Compare with the external data source, not with totals. On a shared account, other items come and go. Ask the external data source the same question and compare.
• Seed the fixture from a script. The seed script creates the Stripe fixture's three customers, three products and their prices in a test-mode account. It creates only what's missing, and looks for existing prices among each product's first 100, so a second run creates nothing and prints {"ok":true,"created":[]}.
• Refuse production credentials in your scripts. The Stripe seed and smoke scripts load the key through a helper that accepts only test-mode keys:

// Shared by the seed and smoke scripts: loads a test-mode key and talks to Stripe directly.
import { readFileSync } from 'node:fs';
import process from 'node:process';

/** Must match `STRIPE_API_VERSION` in collections.ts. The smoke checks that it does. */
export const STRIPE_API_VERSION = '2026-08-26.dahlia';
export const FIXTURE_EMAIL_DOMAIN = 'stripe-fixture.monospace.test';
export const FIXTURE_MARKER = 'stripe-connector-fixture-v1';

/** `STRIPE_API_KEY`, or the file in `STRIPE_API_KEY_FILE`, which holds only the key. Surrounding whitespace is trimmed. */
export function loadApiKey() {
    const file = process.env.STRIPE_API_KEY_FILE;
    const key = process.env.STRIPE_API_KEY?.trim() ?? (file === undefined ? undefined : readFileSync(file, 'utf8').trim());
    if (key === undefined || key === '') throw new Error('Set STRIPE_API_KEY or STRIPE_API_KEY_FILE to a Stripe test-mode key.');
    // These scripts write to the account: never point them at live data.
    if (!/^(?:sk|rk)_test_/u.test(key)) throw new Error('The Stripe scripts only accept a test-mode key.');
    return key;
}

The Engine smoke checks the key itself before it sends any request, to the instance or to Stripe: [[ $KEY =~ ^(sk|rk)_test_ ]] || { echo 'Use a Stripe test-mode key.' >&2; exit 1; }

• Plan for what the external data source can't delete. Stripe can't delete prices, or a product that still has prices. Each run of the Stripe Engine smoke leaves one archived price on prod_msfx_locked; the customer and the three products it creates are deleted.

The seed script looks each fixture item up before it creates it:

// Creates the fixture the smoke scripts read, in a Stripe test-mode account. Idempotent: it
// creates only what is missing, found by email for customers and by id and nickname for products
// and prices. Every fixture object carries `metadata[fixture]` so it can be told apart.
import { FIXTURE_EMAIL_DOMAIN, FIXTURE_MARKER, loadApiKey, stripeApi } from './stripe.mjs';

const stripe = stripeApi(loadApiKey());

const customers = [
    { key: 'ada', name: 'Ada Lovelace', phone: '+4930123456', description: 'Fixture customer with payments' },
    { key: 'grace', name: 'Grace Hopper', phone: '+4930654321', description: 'Fixture customer with a refund and a subscription' },
    { key: 'linus', name: 'Linus Torvalds', description: 'Fixture customer with a draft invoice' },
];

const products = [
    { id: 'prod_msfx_starter', name: 'Fixture Starter', description: 'Starter plan', prices: [
        { nickname: 'msfx_starter_monthly', unit_amount: '900', interval: 'month' },
    ] },
    { id: 'prod_msfx_pro', name: 'Fixture Pro', description: 'Pro plan', prices: [
        { nickname: 'msfx_pro_monthly', unit_amount: '2900', interval: 'month' },
        { nickname: 'msfx_pro_setup', unit_amount: '5000' },
    ] },
    // A product with a price can't be deleted: the smokes use this one for that refusal.
    { id: 'prod_msfx_locked', name: 'Fixture Locked', description: 'Cannot be deleted', prices: [
        { nickname: 'msfx_locked', unit_amount: '100' },
    ] },
];

const created = [];

for (const { key, ...fields } of customers) {
    const email = `${key}@${FIXTURE_EMAIL_DOMAIN}`;
    const [existing] = await stripe.list('customers', { email, limit: '1' });
    if (existing !== undefined) continue;

    await stripe.post('customers', { ...fields, email, 'metadata[fixture]': FIXTURE_MARKER, 'metadata[fixture_key]': key });
    created.push(email);
}

for (const { id, prices, ...fields } of products) {
    const existing = await stripe.get(`products/${id}`).catch((error) => {
        if (error.status === 404) return undefined;
        throw error;
    });

    if (existing === undefined) {
        await stripe.post('products', { id, ...fields, 'metadata[fixture]': FIXTURE_MARKER, 'metadata[fixture_key]': id });
        created.push(id);
    }

    const current = await stripe.list('prices', { product: id, limit: '100' });

    for (const { nickname, unit_amount, interval } of prices) {
        if (current.some((price) => price.nickname === nickname)) continue;

        await stripe.post('prices', {
            product: id,
            currency: 'eur',
            unit_amount,
            nickname,
            ...(interval === undefined ? {} : { 'recurring[interval]': interval }),
            'metadata[fixture]': FIXTURE_MARKER,
            'metadata[fixture_key]': nickname,
        });
        created.push(nickname);
    }
}

process.stdout.write(`${JSON.stringify({ ok: true, created })}\n`);

Emulators

An emulator is a state fixture, not a model of the real API. Design against the vendor's documentation and live probes, and use the emulator for state only.

• Authentication: an emulator that accepts any token can't test a rejected credential.
• Parameters: an emulator that doesn't implement a parameter can ignore it and answer with unfiltered items. A pushdown bug then returns the same items as correct code.
• Paging and shapes: paging, timestamps, response shapes, and rate-limit statuses can differ from the real API.

Cover what the emulator can't show with wire assertions and a stand-in.

1. Lint, Typecheck, and Run Unit Tests

Run the linter, tsc --noEmit, and your unit tests. The build bundles without typechecking, so a type error reaches the artifact unnoticed.

npm run lint

pnpm lint

yarn lint

bun run lint

npm run typecheck

pnpm typecheck

yarn typecheck

bun run typecheck

npm run test

pnpm test

yarn test

bun run test

The example projects lint with ESLint's recommended rules and typescript-eslint's strict and stylistic type-checked rules:

import js from '@eslint/js';
import { defineConfig } from 'eslint/config';
import globals from 'globals';
import tseslint from 'typescript-eslint';

export default defineConfig(
    { ignores: ['dist/'] },
    js.configs.recommended,
    {
        // The smoke scripts run in Node.
        files: ['scripts/**/*.mjs'],
        languageOptions: { globals: globals.node },
    },
    {
        files: ['**/*.ts'],
        extends: [tseslint.configs.strictTypeChecked, tseslint.configs.stylisticTypeChecked],
        languageOptions: { parserOptions: { projectService: true, tsconfigRootDir: import.meta.dirname } },
        // Numbers read fine in messages; the rule still catches objects and `undefined`.
        rules: { '@typescript-eslint/restrict-template-expressions': ['error', { allowNumber: true }] },
    },
);

Unit tests import your sources, not the build, and run without the network. The Stripe connector has three test files:

connector.test.ts records every request and lets each test decide the answer. That covers what a live account can't produce on demand:

// Network-free tests through the connector's handlers, with `fetch` replaced by a fake Stripe.
// They cover what a live account can't produce on demand: timeouts, bad bodies, rate limits, and
// a bulk write that fails partway.
import type { FilterSerialized, OperationSerialized, ReadManyOperationSerialized } from '@monospace/extension-kit/data-connector';
import {
    AccessDenied,
    ConnectionFailed,
    ForeignKeyConstraintViolation,
    InvalidConfiguration,
    QueryRejected,
    RateLimited,
} from '@monospace/extension-kit/data-connector';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import connector from '../src/data-connectors/stripe/index';

interface SentRequest {
    readonly method: string;
    /** Below `/v1/`. */
    readonly path: string;
    readonly search: URLSearchParams;
    readonly form: URLSearchParams;
}

let sent: SentRequest[];
let answer: (request: SentRequest) => Response | Promise<Response>;

beforeEach(() => {
    sent = [];
    answer = () => Response.json({ data: [], has_more: false });

    vi.stubGlobal('fetch', async (url: URL, init: { method: string; body?: string }) => {
        const request = { method: init.method, path: url.pathname.replace('/v1/', ''), search: url.searchParams, form: new URLSearchParams(init.body) };
        sent.push(request);
        return await answer(request);
    });

    return () => vi.unstubAllGlobals();
});

const API_KEY = 'sk_test_fake';

async function handlers(config: Record<string, unknown> = {}) {
    return await connector.setup({ config: { apiKey: API_KEY, ...config } });
}

async function query(operation: OperationSerialized, config?: Record<string, unknown>) {
    const connection = await handlers(config);
    return await connection.query({ query: operation });
}

function customer(id: string, created = 1_767_225_600, fields: Record<string, unknown> = {}) {
    return { id, object: 'customer', name: id, email: null, phone: null, description: null, balance: 0, created, ...fields };
}

function stripeError(status: number, error: Record<string, string>): Response {
    return Response.json({ error: { ...error, request_log_url: 'https://dashboard.stripe.com/acct_123/test/logs/req_1', doc_url: 'https://docs' } }, { status });
}

A test can then hold some writes in flight and fail another, to check that a bulk delete stops starting new writes after the first failure:

    it('stops a bulk delete after the first failure, lets the writes in flight finish, and says how far it got', async () => {
        const ids = ['cus_1', 'cus_2', 'cus_3', 'cus_4', 'cus_5', 'cus_6'];
        let release: () => void = () => undefined;
        const inFlight = new Promise<void>((resolve) => {
            release = resolve;
        });

        answer = async (request) => {
            // Newest first follows the ids' order, so `cus_1` is deleted first.
            if (request.method === 'GET') return Response.json(customer(request.path.split('/')[1] ?? '', 100 - ids.indexOf(request.path.split('/')[1] ?? '')));
            if (request.path === 'customers/cus_1') {
                setTimeout(release, 10);
                return stripeError(400, { type: 'invalid_request_error', message: 'Stripe refused' });
            }

            await inFlight;
            return Response.json({ id: request.path, deleted: true });
        };

        const filter = { in: { field: 'id' }, values: ids } as const;
        const error = await failure(query({ op: 'deleteMany', collection: 'customers', select: ['id'], filter }));
        expect(error).toBeInstanceOf(QueryRejected);
        expect(error).toHaveProperty('message', 'Deleted 3 of 6 customers before this error: Stripe refused');
        expect(sent.filter((request) => request.method === 'DELETE').map((request) => request.path)).toEqual(['customers/cus_1', 'customers/cus_2', 'customers/cus_3', 'customers/cus_4']);
    });

Tie your declarations to your code with a test. This one plans every operator each collection declares, with one sample value each. It catches a declared operator the plan always refuses. It doesn't prove every value, combination or logical operator plans:

// Network-free tests for `planRequests`: which Stripe requests a filter becomes, and which filters
// are refused before any request.
import type { FilterOperator, FilterSerialized, Value } from '@monospace/extension-kit/data-connector';
import type { ResourceSpec } from '../src/data-connectors/stripe/resources';
import { QueryRejected } from '@monospace/extension-kit/data-connector';
import { describe, expect, it } from 'vitest';
import { schemaDocument } from '../src/data-connectors/stripe/collections';
import { planRequests } from '../src/data-connectors/stripe/plan';
import { findResource } from '../src/data-connectors/stripe/resources';

// …
describe('declarations and plan agree', () => {
    // Every operator a collection declares must plan for a sample value, so no declared operator is one the plan always refuses.
    const samples: Record<string, Value> = { id: 'x_1', email: 'a@x.test', productId: 'prod_1', active: true, type: 'recurring', created: '2026-01-01T00:00:00Z' };
    const declared = (schemaDocument(true).collections ?? []).flatMap((collection) => Object.entries(collection.operations.readMany?.filter?.fields ?? {})
        .flatMap(([field, { operators }]) => Object.keys(operators).map((operator) => [collection.name, field, operator] as const)));

    const comparisons: readonly FilterOperator[] = ['equals', 'lessThan', 'lessThanOrEquals', 'greaterThan', 'greaterThanOrEquals'];

    it.each(declared)('%s: `%s` with `%s` plans', (collection, field, operator) => {
        const value = samples[field] ?? null;
        const cmp = comparisons.find((candidate) => candidate === operator);
        const filter: FilterSerialized = cmp === undefined ? { in: { field }, values: [value] } : { cmp, left: { field }, right: { value } };
        expect(() => planRequests(resource(collection), filter)).not.toThrow();
    });
});

Unit tests share your copy of the kit, so instanceof works in them.

When it works, the linter and tsc exit 0, and Vitest reports every test passed (Tests 110 passed (110) for the Stripe connector).

2. Build

Build the artifact with the CLI. The build parses the bundled output and checks that it has a default export and no import, export … from, dynamic import() or direct require(...) left. It doesn't run your code, and a require it can't see (e.g., one called with spread arguments) passes. See Data Connector Bundles.

npm run build

pnpm build

yarn build

bun run build

example/stripe-connector  example/stripe-connector.js
Artifact                  /home/you/stripe/dist

With --json, check that outcome is ok. See Build an Extension for the build steps and errors.

When it works, dist/ holds manifest.json and one JavaScript file per entry (e.g., dist/example/stripe-connector.js).

3. Run an Artifact Smoke

Import the built file in Node and call setup and the handlers directly. No Engine, no harness: the default export is your connector definition.

npm run seed

pnpm seed

yarn seed

bun run seed

npm run smoke

pnpm smoke

yarn smoke

bun run smoke

The Stripe smoke wraps fetch first, so every check can see the requests to the external data source, and can change Stripe between two of them. Then it imports the artifact:

// Artifact smoke: imports the built connector and drives it against a Stripe test account.
// Expectations come from Stripe's own answers or the namespaced fixture (`npm run seed`), never
// account totals. Run `npm run build` first.
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import { ErrorCode } from '@monospace/extension-kit/data-connector';
import { FIXTURE_EMAIL_DOMAIN, loadApiKey, STRIPE_API_VERSION, stripeApi } from './stripe.mjs';

const apiKey = loadApiKey();
const stripe = stripeApi(apiKey);
const requests = [];
const realFetch = globalThis.fetch;
/** Runs once, after the next request's answer arrives: lets a check change Stripe between two requests. */
let afterNextRequest;

globalThis.fetch = async (url, init) => {
    requests.push({ method: init.method, url: new URL(url), version: init.headers['Stripe-Version'], signal: init.signal });
    const response = await realFetch(url, init);
    const hook = afterNextRequest;
    afterNextRequest = undefined;
    await hook?.();
    return response;
};

const manifest = JSON.parse(await readFile(new URL('../dist/manifest.json', import.meta.url), 'utf8'));
const { default: descriptor } = await import(new URL('../dist/example/stripe-connector.js', import.meta.url).href);
const connector = await descriptor.setup({ config: { apiKey } });
const checks = [];

async function check(name, run) {
    requests.length = 0;
    await run();
    checks.push(name);
}

async function query(operation, target = connector) {
    return await target.query({ query: operation });
}

async function read(collection, options = {}) {
    const result = await query({ op: 'readMany', collection, select: ['id'], ...options });
    return result.records;
}

async function ids(collection, options) {
    const records = await read(collection, options);
    return records.map((record) => record.id);
}

function compare(cmp, field, value) {
    return { cmp, left: { field }, right: { value } };
}

• Test the built file, not your sources. The artifact is what Engine loads.
• Name every check. The script ends by printing { ok, checks }, so a run shows what it covered.
• Compare error codes, not classes. The artifact bundles its own copy of the kit, so instanceof against your import fails. Read error.monospace.code. See Check What Callers See.

scripts/stripe.mjs is a small Stripe client of its own. It doesn't import the connector, so comparing the two proves something.

Cover These in the Artifact Smoke

These checks from the Stripe smoke cover configuration, credentials, and keyed operations:

await check('setup validates the configuration with InvalidConfiguration, without a request', async () => {
    assert.equal(await codeOf(() => descriptor.setup({ config: {} })), ErrorCode.InvalidConfiguration);
    assert.equal(await codeOf(() => descriptor.setup({ config: { apiKey: 'pk_test_123' } })), ErrorCode.InvalidConfiguration);
    assert.equal(await codeOf(() => descriptor.setup({ config: { apiKey, enableWrites: 'yes' } })), ErrorCode.InvalidConfiguration);
    assert.equal(await codeOf(() => descriptor.setup({ config: { apiKey, maxItemsPerQuery: 0 } })), ErrorCode.InvalidConfiguration);
    assert.equal(requests.length, 0);
});

await check('a wrong key fails testConnection with AccessDenied, without echoing the key', async () => {
    const wrong = await descriptor.setup({ config: { apiKey: 'sk_test_wrongwrongwrong' } });
    const error = await wrong.testConnection().catch((thrown) => thrown);
    assert.equal(error.monospace?.code, ErrorCode.AccessDenied);
    assert.equal(error.message, 'Stripe rejected the API key (HTTP 401).');
    assert.deepEqual(await connector.testConnection(), { ok: true });
});
// …
await check('readOne: key, guard hit; a guard miss, an unknown key and an impossible id answer record null', async () => {
    const one = async (key, filter) => await query({ op: 'readOne', collection: 'customers', select: ['id', 'name'], key, filter });
    assert.deepEqual(await one({ id: ada.id }), { record: { id: ada.id, name: 'Ada Lovelace' } });
    assert.deepEqual(await one({ id: ada.id }, compare('equals', 'email', fixtureEmail('ada'))), { record: { id: ada.id, name: 'Ada Lovelace' } });
    assert.deepEqual(await one({ id: ada.id }, compare('equals', 'email', fixtureEmail('grace'))), { record: null });
    assert.deepEqual(await one({ id: 'cus_doesnotexist' }), { record: null });
    requests.length = 0;
    assert.deepEqual(await one({ id: '..' }), { record: null });
    assert.equal(requests.length, 0);
});

The helpers come from the top of the script: query sends { query: operation } to the handler, read and ids return what a readMany answers, compare builds a comparison filter, and codeOf returns the monospace.code of whatever a call throws, or PLAIN_ERROR for a plain Error. The checks compare with ErrorCode, never a hard-coded number. ada is a fixture customer the script looks up before the first check.

The artifact smoke runs in Node, while Engine runs the connector in a sandboxed Deno worker. Network permissions, the timeouts on loading and setup, and Engine's own requests exist only once Engine runs the connector, from rung 5 on.

When it works, the smoke prints {"ok":true,"checks":[…]} with every check named:

{"ok":true,"checks":["manifest declares exactly net:api.stripe.com and the three returning capabilities, and there is no preflight","setup validates the configuration with InvalidConfiguration, without a request","a wrong key fails testConnection with AccessDenied, without echoing the key","every request carries the pinned API version and an abort signal","drift: every declared field reads the value at its Stripe path","the first page and an offset match Stripe","limit 150 is a Stripe page of 100 and the next 50","email in: one list request per value, merged newest first","prices by productId match Stripe, with camelCase fields","a filter on a product that does not exist answers no item","created: a fraction of a second moves the bound to the next whole second","readOne: key, guard hit; a guard miss, an unknown key and an impossible id answer record null","refusals send no request: undeclared shapes are plain errors, the rest rejected","an unbounded read beyond maxItemsPerQuery is refused","writes are refused when enableWrites is false","customer create, updateOne (a guard miss writes nothing and answers record null), deleteOne","updateOne and deleteOne answer record null when the customer is deleted after the connector retrieved it","a price for a product that does not exist is a ForeignKeyConstraintViolation","deleting a product that has prices is a ForeignKeyConstraintViolation"]}

Assert What Goes on the Wire

A result check passes whenever the items are right. Two things make items right even when the request to the external data source is wrong: a fixture that ignores a parameter, and the connector's own re-check of returned items against the filter. The Stripe connector re-checks merged and retrieved results, not the answer of a single list request. Record the outgoing requests, and assert on them.

The Stripe smoke checks that both requests of an email in read carry the pinned API version and an abort signal, and that a created bound one microsecond past a customer's created second reaches Stripe as the next whole second:

await check('every request carries the pinned API version and an abort signal', async () => {
    await ids('customers', { filter: { in: { field: 'email' }, values: ['ada', 'grace'].map((key) => fixtureEmail(key)) } });
    assert.equal(requests.length, 2);

    for (const request of requests) {
        assert.equal(request.version, STRIPE_API_VERSION);
        assert.ok(request.signal instanceof AbortSignal);
    }
});
// …
await check('created: a fraction of a second moves the bound to the next whole second', async () => {
    const created = new Date(ada.created * 1000).toISOString();
    const justAfter = created.replace('.000Z', '.000001Z');
    const byEmail = compare('equals', 'email', fixtureEmail('ada'));
    assert.deepEqual(await ids('customers', { filter: { and: [byEmail, compare('greaterThanOrEquals', 'created', created)] } }), [ada.id]);
    assert.deepEqual(await ids('customers', { filter: { and: [byEmail, compare('greaterThanOrEquals', 'created', justAfter)] } }), []);
    assert.equal(new URL(requests.at(-1).url).searchParams.get('created[gte]'), String(ada.created + 1));
    assert.deepEqual(await ids('customers', { filter: { and: [byEmail, compare('lessThan', 'created', justAfter)] } }), [ada.id]);
});

Assert absence too. A refusal decided by the request's shape must send no request, and a result check can't tell. These Stripe refusals are all shape-based. The first three name shapes the collection doesn't declare, so they're plain errors. An extension update without a schema refresh can send an undeclared sort or count. A permission rule's condition can also carry an undeclared filter field; the null filter is a valid request Stripe can't express, so it's QueryRejected. The last two add top-level ids: the undeclared field inside a not is still a plain error, and a malformed timestamp behind an or that an item without values already decides is still refused before the retrieve:

await check('refusals send no request: undeclared shapes are plain errors, the rest rejected', async () => {
    assert.equal(await codeOf(() => ids('customers', { filter: compare('equals', 'name', 'Ada Lovelace') })), PLAIN_ERROR);
    assert.equal(await codeOf(() => ids('customers', { sort: [{ field: 'created', direction: 'ascending' }] })), PLAIN_ERROR);
    assert.equal(await codeOf(() => query({ op: 'readMany', collection: 'customers', count: true, limit: 0 })), PLAIN_ERROR);
    assert.equal(await codeOf(() => ids('customers', { filter: compare('equals', 'email', null) })), ErrorCode.QueryRejected);
    // With top-level ids too: every comparison is checked, even one an `or` or `not` would skip.
    const byId = compare('equals', 'id', ada.id);
    assert.equal(await codeOf(() => ids('customers', { filter: { and: [byId, { not: compare('equals', 'name', 'x') }] } })), PLAIN_ERROR);
    assert.equal(await codeOf(() => ids('customers', { filter: { and: [byId, { or: [compare('equals', 'email', null), compare('equals', 'created', 'bad-time')] }] } })), ErrorCode.QueryRejected);
    assert.equal(requests.length, 0);
});

A wrapped fetch sees only requests made in your Node process. It can't see requests the connector makes inside Engine. For that, put a recording HTTP proxy or a stand-in in front of the external data source.

The blind spot: you write the expected parameters, so a wrong belief about the external data source passes. Pin each fact about it you rely on with a live probe.

Compare with the External Data Source

Ask the external data source directly, through your independent client, and compare. This proves paging and order without hard-coded totals. The check needs more than 100 customers in the account for a second page, and at least 150 to fill the whole limit:

await check('limit 150 is a Stripe page of 100 and the next 50', async () => {
    const first = await stripe.list('customers', { limit: '100' });
    const next = await stripe.list('customers', { limit: '50', starting_after: first.at(-1).id });
    requests.length = 0;
    assert.deepEqual(await ids('customers', { limit: 150 }), [...first, ...next].map((customer) => customer.id));
    assert.equal(requests.length, 2);
});

A comparison proves nothing about a filter the fixture ignores: the external data source's answer is then equally wrong. Pair it with wire assertions.

Check Null Handling

Engine sends filters your callers didn't type, such as not around an update's field rules. Where a nullable field declares equals and its filter declares not, a caller's _null: true arrives as equals with null, and _null: false as not around it. Keep at least one fixture item with nulls in filterable fields, and check:

• field = null matches only items where the field is null.
• not (field = x) doesn't match an item where field is null. Under SQL's three-valued logic, that comparison is unknown, not true.

The Stripe connector's test/filter.test.ts checks both against an item with a null email. Your connector defines its own null handling, so test the rules it implements. See Handle Requests You Didn't Declare.

4. Install and Check the Catalog

Install the build into your instance (your running Monospace deployment), then confirm that the instance lists your connector. A bundle can load without contributing a connector, for example when its entry id is already taken. Only a warning in the Engine log reports it.

A listed entry proves the instance registered the entry from a valid manifest. It doesn't prove that the connector file loads, or that setup or introspect works; rung 5 checks those.

npm run build -- --install-to ../monospace/extensions

pnpm build --install-to ../monospace/extensions

yarn build --install-to ../monospace/extensions

bun run build --install-to ../monospace/extensions

With MONOSPACE_EXTENSIONS__AUTO_RELOAD on, the instance reloads the extension once its files have been quiet for 500 ms. Otherwise, restart it. See Install and Manage Extensions.

Then list the workspace's data connectors with GET /api/{workspace}/connectors/data:

{
  "data": [
    {
      "id": "example/stripe-connector",
      "version": "0.1.0",
      "name": "Stripe",
      "icon": "simple-icons:stripe",
      "extensionId": "example/stripe",
      "permissions": [
        {
          "permission": "net:api.stripe.com",
          "reason": "Read and write customers, products and prices through the Stripe API"
        }
      ]
    }
  ]
}

In scripts, poll until your entry id appears with your extensionId, instead of sleeping for a fixed time. The catalog reports the version from your manifest, so it changes only when you bump the version. An entry that's already listed can still be the previous build: bump the version, or check behavior the rebuild changed, to confirm the new build is active.

When it works, the catalog lists your entry id, with the version and permissions from your manifest.

5. Run an Engine Smoke

Drive the installed connector through Engine's REST API. This is the only rung that runs Engine's declaration rules, the permission handshake, the sandbox, and the requests Engine composes on its own.

Run it in a fresh data source every time. A reload swaps a data source's code but keeps the schema it was created with. Creating a data source checks your current declarations, and so does reintrospecting one. The example scripts create a throwaway workspace per run and delete it at the end, which also avoids name collisions such as customers_2.

Send Cache-Control: no-cache on every read in your own smoke. Otherwise a repeated read can come from the query-result cache without reaching your connector, and pass when the connector would fail.

The example scripts need curl and jq, and read these settings from the environment:

npm run smoke:engine

pnpm smoke:engine

yarn smoke:engine

bun run smoke:engine

A shared library signs in, sends each request, and checks it. req sends one request and prints it with the HTTP status and the response. expect checks the last status, then compares each jq expression's result with an expected JSON value, and stops the run at the first mismatch. A trap tries to delete every throwaway workspace when the script exits, even after a failed check. That cleanup is best effort: it ignores failures. A successful run also deletes its workspaces as checked steps:

# Shared by the Engine smoke scripts. Sends requests to a running Engine, prints each request and
# response as Markdown (the access token as $TOKEN), and stops the run at the first failed check.
# Needs curl and jq. Settings: ENGINE_URL, ENGINE_EMAIL, ENGINE_PASSWORD.
set -euo pipefail

ENGINE_URL=${ENGINE_URL:-http://localhost:8100}
REDACT=${REDACT:-}
CHECKS=0
WORKSPACES=()

TOKEN=$(curl -sf -X POST "$ENGINE_URL/api/auth/providers/local/password/login" \
    -H 'Content-Type: application/json' \
    -d "{\"email\":\"${ENGINE_EMAIL:-monospace@example.com}\",\"password\":\"${ENGINE_PASSWORD:-monospace}\",\"mode\":\"json\"}" |
    jq -r .accessToken)

say() { printf '\n%s\n' "$*"; }

fail() {
    printf '\nFAIL: %s\n' "$*" >&2
    exit 1
}

# Replaces the secret in REDACT, when set, so it never reaches the output.
redact() {
    if [ -n "$REDACT" ]; then printf '%s' "${1//"$REDACT"/sk_test_YOUR_KEY}"; else printf '%s' "$1"; fi
}

# req METHOD PATH [BODY] [JQ_FILTER]: sends one request, prints it and the (filtered) response.
# `Cache-Control: no-cache` makes Engine run every read, so no check passes on a cached result.
req() {
    local method=$1 path=$2 body=${3:-} filter=${4:-.} out
    printf '\n```bash\ncurl -g -X %s "%s%s" \\\n  -H "Authorization: Bearer $TOKEN" \\\n  -H "Cache-Control: no-cache"' "$method" "$ENGINE_URL" "$path"
    if [ -n "$body" ]; then
        printf ' \\\n  -H "Content-Type: application/json" \\\n  -d '"'"'%s'"'"'' "$(redact "$body")"
        out=$(curl -sg -X "$method" "$ENGINE_URL$path" -H "Authorization: Bearer $TOKEN" -H 'Cache-Control: no-cache' -H 'Content-Type: application/json' -d "$body" -w '\n%{http_code}')
    else
        out=$(curl -sg -X "$method" "$ENGINE_URL$path" -H "Authorization: Bearer $TOKEN" -H 'Cache-Control: no-cache' -w '\n%{http_code}')
    fi
    LAST_CODE=$(printf '%s' "$out" | tail -n 1)
    LAST_BODY=$(redact "$(printf '%s' "$out" | sed '$d')")
    printf '\n```\n\n```json\n// HTTP %s\n' "$LAST_CODE"
    if [ -z "$LAST_BODY" ]; then echo '// (empty body)'; else printf '%s' "$LAST_BODY" | jq "$filter" 2>/dev/null || printf '%s\n' "$LAST_BODY"; fi
    printf '```\n'
}

# expect STATUS [JQ_EXPRESSION EXPECTED_JSON]...: checks the last response's status, then each
# expression's compact JSON result against the expected value.
expect() {
    local status=$1 checked actual
    shift
    [ "$LAST_CODE" = "$status" ] || fail "expected HTTP $status, got $LAST_CODE: $LAST_BODY"
    checked="HTTP $status"
    while [ $# -ge 2 ]; do
        actual=$(printf '%s' "$LAST_BODY" | jq -c "$1")
        [ "$actual" = "$2" ] || fail "expected \`$1\` to be $2, got $actual"
        checked="$checked; \`$1\` is \`$2\`"
        shift 2
    done
    CHECKS=$((CHECKS + 1))
    printf '\nChecked: %s\n' "$checked"
}

# workspace NAME: creates a throwaway workspace that is deleted when the script exits, even on failure.
workspace() {
    WORKSPACES+=("$1")
    req POST /api/system/workspaces "{\"apiName\":\"$1\"}"
    expect 200 '.data.id | type' '"string"'
}

cleanup() {
    local name
    # The `+` form keeps an empty array from failing `set -u` in older bash.
    for name in ${WORKSPACES[@]+"${WORKSPACES[@]}"}; do
        curl -sg -o /dev/null -X DELETE "$ENGINE_URL/api/system/workspaces/$name?fields=id" -H "Authorization: Bearer $TOKEN" || true
    done
}
trap cleanup EXIT

finish() {
    say "All $CHECKS checks passed."
}

The Stripe script starts by refusing any key that isn't test mode, then walks through the steps below:

#!/usr/bin/env bash
# Engine smoke for the Stripe connector. Needs a running Engine with the bundle installed, the
# fixture from `npm run seed`, and a test-mode key in STRIPE_API_KEY (or a file in
# STRIPE_API_KEY_FILE). Prints Markdown with the key redacted, and exits non-zero on the first
# failed check. Leaves one archived price on `prod_msfx_locked` per run: Stripe can't delete prices.
KEY=${STRIPE_API_KEY:-$(cat "${STRIPE_API_KEY_FILE:?set STRIPE_API_KEY or STRIPE_API_KEY_FILE}")}
# The script writes to the account: never point it at live data. Checked before any request.
[[ $KEY =~ ^(sk|rk)_test_ ]] || { echo 'Use a Stripe test-mode key.' >&2; exit 1; }
REDACT=$KEY
. "$(dirname "$0")/engine-lib.sh"

STAMP=$(date +%s)
WS=stripe-$STAMP
READ_ONLY_WS=stripe-ro-$STAMP
PROVIDER=example/stripe-connector
PERMISSIONS='[{"permission":"net:api.stripe.com","reason":"Read and write customers, products and prices through the Stripe API"}]'
FIXTURE=stripe-fixture.monospace.test

CONFIG="{\"apiKey\":\"$KEY\"}"
# Read-only, and with a budget of two Stripe objects per read, to see a QueryRejected through Engine.
READ_ONLY_CONFIG="{\"apiKey\":\"$KEY\",\"enableWrites\":false,\"maxItemsPerQuery\":2}"

# source_body CONFIGURATION [API_NAME]: a data source for this connector, with the approved permissions.
source_body() {
    if [ -n "${2:-}" ]; then
        printf '{"apiName":"%s","provider":"%s","config":%s,"permissions":%s}' "$2" "$PROVIDER" "$1" "$PERMISSIONS"
    else
        printf '{"provider":"%s","config":%s,"permissions":%s}' "$PROVIDER" "$1" "$PERMISSIONS"
    fi
}
# Stripe's own answer, to compare Engine's answer with.
stripe() { curl -sfg "https://api.stripe.com/v1/$1" -u "$KEY:" -H 'Stripe-Version: 2026-08-26.dahlia'; }

say "### Create a throwaway workspace"
workspace "$WS"

say "### Handshake: without permissions, Engine answers 7001 with the set to approve"
req POST "/api/$WS/sources/data/test" "{\"provider\":\"$PROVIDER\",\"config\":$CONFIG}"
expect 422 '.code' '"7001"' '.meta.permissions' "$PERMISSIONS"

say "### A well-formed wrong key: setup accepts it, testConnection gets Stripe's 401 (AccessDenied)"
req POST "/api/$WS/sources/data/test" "$(source_body '{"apiKey":"sk_test_wrongwrongwrong"}')"
expect 422 '.source.message' '"Invalid extension configuration"' \
    'tostring | contains("Stripe rejected the API key (HTTP 401).")' 'true' 'tostring | contains("wrong")' 'false'

say "### A malformed key: setup refuses it with InvalidConfiguration, which fails the test with 422"
req POST "/api/$WS/sources/data/test" "$(source_body '{"apiKey":"not-a-key"}')"
expect 422 '.message' '"The extension could not be started"' \
    '.source.source.message' '"Invalid extension configuration"' 'tostring | contains("must be a secret")' 'true'

say "### Test with the real test key"
req POST "/api/$WS/sources/data/test" "$(source_body "$CONFIG")"
expect 204

req's optional fourth argument is a jq filter that trims the printed response. expect always checks the whole body, after the configured key is redacted from it. LAST_BODY holds the last response, so a script can read an id from it for the next request.

Work through these steps in order:

{
  "message": "The connector requires permissions this data source was not granted. Approve them and retry.",
  "code": "7001",
  "meta": {
    "permissions": [
      {
        "permission": "net:api.stripe.com",
        "reason": "Read and write customers, products and prices through the Stripe API"
      }
    ]
  }
}

say "### A bulk delete that fails partway says how far it got"
for SUFFIX in a b c; do
    req POST "/api/$WS/items/products?fields=id" "{\"id\":\"prod_docs_smoke_${STAMP}_$SUFFIX\",\"name\":\"Docs Smoke $SUFFIX\"}"
    expect 200 '.data[0].id' "\"prod_docs_smoke_${STAMP}_$SUFFIX\""
done
IDS="filter[id][_in][]=prod_docs_smoke_${STAMP}_a&filter[id][_in][]=prod_docs_smoke_${STAMP}_b&filter[id][_in][]=prod_docs_smoke_${STAMP}_c&filter[id][_in][]=prod_msfx_locked"
req DELETE "/api/$WS/items/products?fields=id&$IDS"
expect 422 'tostring | contains("Deleted 3 of 4 products before this error: This product cannot be deleted")' 'true'
req GET "/api/$WS/items/products?fields=id&$IDS"
expect 200 '[.data[].id]' '["prod_msfx_locked"]'

When it works, every check passes and the script ends with its count (All 47 checks passed. for the Stripe smoke). The data source is created from a fresh workspace, and every relation you declared appears in the OpenAPI document.

6. Break Each Check on Purpose

A check that can't fail proves nothing. Right after you write a check, break the rule it covers, rebuild, rerun, and confirm that this specific check fails. Then restore the code, confirm the restore with a diff, and rebuild.

Each mutation below turns a specific check in the example connectors red:

These cover a sample of the checks, not every one. Break each new check once, when you write it.

Mutations worth trying on any connector:

• Reads: ignore offset; stop following the external data source's next page; reverse the order.
• Filters: drop the re-check of returned items; treat not unknown as true; drop a null guard.
• Pushdown: move a translated bound by one unit of the external data source's precision (e.g., one second).
• Keyed operations: ignore the guard.
• Answers: omit a selected field, or answer a keyed operation with records. Engine fails both, so the Engine smoke must go red.
• Refusals and errors: accept a sort direction you don't declare; map every 4xx to the same class.

When a check doesn't go red, add a wire assertion, a stand-in, or more fixture data until it does.

When it works, each mutation turns a named check red, and the restored code passes again.

7. Run the Ladder Twice

Run rungs 1 to 5 twice, back to back. Residue from the first run, such as an item left by an interrupted delete or a fixture relation a test changed, can fail the second run.

Start each run by deleting leftovers from an interrupted one. The Stripe smoke deletes customers with its smoke email before it creates one:

    const email = fixtureEmail('docs-smoke');
    for (const leftover of await stripe.list('customers', { email })) await stripe.delete(`customers/${leftover.id}`);

stripe.list reads only the first page. When leftovers can outnumber a page, follow every page.

When it works, both runs pass with the same checks and the same counts.

8. Try It in Studio

Create the data source in Studio and use it the way your users will. No example connector automates this rung.

1. Create a data source from your connector: Data Model > Add Source. Enter the configuration as JSON under Connection Parameters (e.g., { "apiKey": "sk_test_…" } for Stripe), and review the hosts Studio lists under Permissions. See Add a Data Source in Studio.
2. Open each collection. Browse, filter, sort, and page with every control Studio offers.
3. Create, edit, and delete items where the contract allows writes.
4. Test a data source with a wrong credential.

Studio's controls come from what Engine accepts, which includes requests your connector refuses at runtime (e.g., an unfiltered read it can't serve, or a page past a search window).

When it works, every collection loads, and every control either works or fails with a message a user can act on.

Checks the Examples Leave Out

The example test suites don't include these checks. Add them where your connector depends on them:

• Live rate limits and outages. The unit tests cover the mapping of 429, 5xx, timeouts and bodies that aren't JSON with a fake fetch. No example script triggers a rate limit or an outage against the live external data source or through Engine. To check a timeout through Engine, install a temporary build with a 1 ms deadline.
• A body that fails while it's read. The timeout tests make fetch itself reject. No test lets the headers arrive and then fails reading the body, the case the examples read inside the same try as fetch for. Give your fake fetch a response whose body stream errors, and one whose body stalls until the deadline aborts it. The smoke checks only that each request carries an abort signal.
• A leaked credential in a response. The Engine smoke redacts the configured key from every response before expect checks it, so it can't catch a response that contains the key. Check the raw responses for your secret.
• What Engine sends, replayed. Record the requests Engine sends your connector at rung 5, and replay them in the artifact smoke. Neither example replays them.
• Request cost. Count requests to the external data source per operation, and fail when an operation exceeds its budget.
• Updates by filter. updateMany is declared, but neither the unit tests nor the smokes exercise it. deleteMany is covered by a unit test and the Engine smoke.

See Also

• Handle Data Connector Errors: map failures of the external data source to error classes, and check what callers see
• Map an External Data Source: probe the external data source and decide what to declare
• Network Permissions: the permission handshake in detail
• CLI: build, install, and watch
• Build Data Connectors with AI Agents: have a coding agent climb the ladder for you