Data Connectors

Handle Data Connector Errors

Map each failure of an external data source to the right error class, refuse requests you can't serve, and keep secrets out of what callers see.
Preview
Extensions are in preview and can change in breaking ways between releases. Use them in non-production environments only. See Stability. Written for Monospace 1.0.0, @monospace/cli 0.2 and @monospace/extension-kit 0.3.

Overview

The external data source is the API, service, or database your data connector extension calls. When a request to it fails, the connector throws one of the kit's error classes where one fits. Engine can then tell a rate limit from a refused write or a rejected credential. Where no class fits, the connector throws a plain Error. The caller, whoever sent the request to the Monospace API through the SDK, REST, or Studio, sees the result: an HTTP status and your error's message.

Work through the seven steps below. They use the error handling of the Stripe connector from Build a Stripe Data Connector as the example. You end with one client that classifies every failure, a deadline on every request, refusals that tell the caller what to change, and checks that prove each mapping. For every class, its code, and the status the caller gets, see Data Connector Errors.

1. Map Failures in One Client

Send every request to the external data source through one function, and map failures there. The rest of the connector then deals in kit errors, and each rule of the external data source (e.g., "only Stripe's resource_missing means a missing object") lives in one place.

In the Stripe connector, #send is the only method that calls fetch. It sets a deadline, reads the whole body inside the same try, and maps a request that gets no answer. The snippets on this page are excerpts. A file's first excerpt includes its imports, and later excerpts of the same file continue it; // … marks the lines an excerpt leaves out. Build a Stripe Data Connector walks through the rest of the connector.

  • fetch rejects when no response arrives (DNS, refused connection, TLS, abort). That's ConnectionFailed.
  • Keep the rejection as cause. fetch also rejects for a host the data source wasn't granted, with an error named NotCapable. Engine looks for it anywhere in the thrown error's cause chain, so ConnectionFailed with { cause: error } still reaches the caller as 422 "Invalid extension permissions", with your message ("The Stripe API could not be reached.") nested inside. Without cause, the same denial reads as an unreachable external data source. See Handle a Denied Request.
  • Read the body inside the same try. A body read can fail or time out too. Inside the try, it becomes ConnectionFailed instead of escaping unbranded.
  • Parse without throwing. The Stripe client's parseJson answers undefined for a body that isn't JSON, and checked turns that into an Error with a readable message, so a proxy's HTML error page never escapes as a SyntaxError.
  • A status is not a rejection. Check response.status yourself; fetch resolves for a 429 or a 500.
  • Throw a kit class for every failure a class describes. An error without one reaches Engine unclassified. From a handler, it's reported as a failed query, and Engine can't tell a rate limit from a refused write. A permission denial in its cause chain still reaches the caller as one. See Know the Defaults for Unbranded Throws.

When it works, every error that leaves a handler names a kit class. The exception is a plain Error you throw on purpose where no class fits (see step 2). No handler lets a fetch, body-read or JSON.parse error escape.

2. Pick a Class for Each Failure

Classify by who can fix the failure, not by the status code alone. Many external data sources use one status for several meanings, so read the error body's code where the external data source has one. The Stripe and Art Institute connectors map failures this way:

FailureThrowNotes
No response: DNS failure, refused connection, TLS error, aborted requestConnectionFailedCatch around fetch, with the rejection as cause, as in step 1
Timeout from AbortSignal.timeoutConnectionFailedThe abort rejects fetch, so the same catch handles it. See step 3
5xxConnectionFailedTreat it as an outage: a later request may work
429RateLimitedPut the wait into the message, as the Stripe client's retryAfter does: "Stripe rate limited the connector. Retry after 2 seconds."
401 or 403: the credential is rejected, or lacks accessAccessDeniedOnly someone with access to the external data source's account can fix it. If your external data source also answers a rate limit with a 403, tell the two apart by what its documentation names (e.g., a header or an error code), and throw RateLimited for the rate limit
400: a value the external data source refuses (e.g., an invalid email)QueryRejectedThe caller can change the value. The Stripe client matches Stripe's type: invalid_request_error and passes Stripe's message on
404 in the external data source's own error format, for a single-item lookupNo error where a missing item means no match. readOne, updateOne and deleteOne answer { record: null }, which the caller sees as 404, and a read by id leaves the item out. Elsewhere, such as a bulk write to an item deleted meanwhile, a plain ErrorNo kit class reports a missing item. See Answer Keyed Misses with record: null
A collection your connector declared no longer exists in the external data source (e.g., a deleted object type)CollectionNotFoundThe caller gets 404. A collection your connector never declared gets a plain Error instead. An extension update without a schema refresh can send one
404 without the external data source's error format (e.g., an HTML page)A plain ErrorA wrong base URL or route. Treating it as "not found" makes a misconfigured data source look empty
400 or 422: a required value is missingNullConstraintViolation
400, 409, or 422: a value must be unique, or the item already existsUniqueConstraintViolation
400, 409, or 422: a referenced item doesn't exist, or another item still references this oneForeignKeyConstraintViolationA write that names a missing item is this: the caller gets 422
Any other failed statusA plain ErrorThe Art Institute connector answers its API's 403 for a page size over 100 this way: the connector sent a request the API refuses, and the caller can't fix that
2xx with a body you can't parse, of the wrong shape, or with a value of the wrong typeA plain ErrorA changed API. Check the shape where you parse it, so it fails in your client instead of deep inside Engine
A valid request your connector can't serve as askedQueryRejectedSee step 4
A request outside what your connector declaresA plain ErrorAn extension update without a schema refresh can send an undeclared collection, operation, sort or count. An undeclared filter comparison can also come from a permission rule's condition. Engine's exempt lookups on keys no declaration can carry aren't in this group: serve them. See step 4
Missing or invalid configurationInvalidConfigurationThrown from setup. See step 5

The examples throw a plain Error when no class fits and nothing the caller changes fixes it, such as a changed API, which retrying doesn't fix either. Engine reports it as a failed query, 500 "Failed to query the database".

checked returns the body of a successful response. toConnectorError maps every other response by Stripe's error code and param, after special cases for rejected keys and rate limits, then by status:

  • param tells a missing object from a missing reference. Stripe names id when the object in the path doesn't exist, and the parameter when a written value refers to a missing object (e.g., a new price's product). The client maps the second to ForeignKeyConstraintViolation, and the first to StripeObjectMissing, its own subclass of Error. A keyed write that races a concurrent delete catches it and answers record: null; anywhere else it reaches Engine as a plain Error.
  • Some failures only mean something for one operation. Stripe refuses to delete a product that still has prices with a 400 that has no code. The client narrows it to ForeignKeyConstraintViolation only for a DELETE whose message names prices. Every other 400 with type: invalid_request_error is a QueryRejected.

Unit tests that import your sources can check the class with instanceof, because they share your copy of the kit. Tests of the built file read monospace.code instead. See step 7.

cause stays on the JavaScript error, where Engine looks only for a permission denial. Engine receives only the code and the message, so put anything the caller needs in the message.

Answer Keyed Misses with record: null

For readOne, updateOne, and deleteOne, answer { record: null } when the key or the guard matches nothing, and write nothing. A miss isn't an error, so don't throw. Engine reports "No record found" to the caller with a 404. Don't answer { records: [] }: Engine fails a keyed operation that answers records with 500.

The Stripe client's retrieve answers undefined for a 404 whose body carries Stripe's resource_missing code, and for a deleted object that Stripe still returns with deleted: true. An id Stripe can't have answers undefined without a request. The connector's findByKey also answers undefined for a guard miss, and each keyed operation answers undefined as record: null:

src/data-connectors/stripe/client.ts
    /** `undefined` when the object doesn't exist or was deleted. */
    async retrieve(path: string, id: string): Promise<StripeObject | undefined> {
        // An id Stripe can't have names no object: answer without a request.
        if (!STRIPE_ID.test(id)) return undefined;

        const response = await this.#send('GET', `${path}/${id}`);
        // Only Stripe's own answer means the object doesn't exist. Any other 404, such as a proxy's
        // HTML page, is an error.
        if (response.status === 404 && readError(response.body).code === 'resource_missing') return undefined;

        const body = checked(response, 'GET', path);
        // Deleted customers and products stay retrievable, as tombstones.
        if (isObject(body) && body.deleted === true) return undefined;

        const object = parseObject(body);
        if (object.id !== id) throw new Error(`Stripe answered a request for \`${id}\` with another object.`);
        return object;
    }

A filter that names an item that doesn't exist is also no match, not an error. Stripe answers prices?product= with a missing product as resource_missing naming the product parameter, and the connector returns no prices for it. It does this only on the first page, and only when param names one of the request's own list parameters; the same error on a later page throws StripeObjectMissing, a plain Error to Engine.

In the Stripe connector, the guard is a read before the write, not a conditional write. Another writer can change the item between the two requests. See Bulk Writes Are Only as Atomic as the External Data Source.

When it works, a guard miss over REST answers 404 with "No record found", and nothing is written.

Retry Only What Is Safe to Repeat

Engine doesn't retry a failed connector call. A 429 from the external data source fails the caller's request unless your connector retries.

  • Reads: retry a small, bounded number of times, and never wait longer than the caller will.
  • Writes: retry only when the external data source supports idempotency keys. Generate one key per item, and reuse it on every retry. A failed write can leave earlier writes applied, and Engine rolls nothing back.
    The Stripe connector creates items one after another, and runs updates and deletes by filter with at most four in flight. After the first failure it starts no new write and lets the writes in flight finish. Then it says how far it got: "Deleted 3 of 4 products before this error: …". See No Transactions.
  • Everything else: throw RateLimited, and put the wait into the message. The message is all the caller gets: Engine answers RateLimited without a Retry-After header. Both example connectors append "Retry after N seconds." when Retry-After is a whole number of seconds, and nothing for an HTTP-date.

See No Automatic Retries.

3. Add a Timeout to Every Request

Engine sets no deadline on query, introspect, or testConnection. An external data source that never answers holds the caller's request open until the caller gives up. Studio's connection test, for example, stops waiting after 60 seconds.

Pass signal: AbortSignal.timeout(...) in the fetch options, as the Stripe client's #send in step 1 does with TIMEOUT_MS (10 seconds). When the deadline passes before the response arrives, fetch rejects with an error named TimeoutError, and the catch throws ConnectionFailed: "The Stripe API did not answer within 10 seconds."

  • The signal also covers the body. If the deadline passes while you read the body (response.text() or response.json()), the read rejects. Read the body inside the same try as fetch, as #send does, or the abort escapes as an unbranded error. The catch throws ConnectionFailed for any rejection, so a body-read timeout fails the same way.
  • The deadline works in the sandbox. Built with a 1 ms deadline, the Stripe connector fails the connection test with "The Stripe API did not answer within 0.001 seconds."
  • A per-call deadline doesn't bound the operation. A read that pages through the external data source makes several calls, each with its own timeout, so it can run for several deadlines. To bound the whole operation, also cap its calls (e.g., with a read budget, as in step 4), or pass one signal to every call it makes.

See Per-Request Deadline and HTTP Through fetch.

When it works, pointing the connector at an external data source that doesn't answer fails the request with the connection error after your deadline instead of hanging.

4. Refuse Requests You Can't Serve

Engine sends only shapes your operations contract declares, plus a few it composes on its own. Some of those requests still can't be served by a given external data source (e.g., an offset past its paging depth). Refuse them with the class that names whose request it is:

  • QueryRejected for a valid request the external data source can't serve as asked: beyond a page window or a read budget, without a filter the external data source requires, or with a shape Engine composes on its own that the external data source can't express (e.g., not around an update's field rules, or or across permission rules). The caller can change the request, so the message says how. Engine answers it with 422.
  • A plain Error for a request outside your declaration. That's an unknown collection or namespace, an operation, a filter field or operator, a sort, a count, a key other than the declared one, or a write. Engine checks callers' requests against the data source's stored schema, but not the filters it sends. So an undeclared comparison can reach your connector from a permission rule's condition or an extension update that narrowed the declaration without a schema refresh. Refuse it rather than drop it: a dropped condition can widen what the caller may touch. The kit has no class for it. Engine answers the plain Error with 500 "Failed to query the database".
  • No error for Engine's relation and read-back lookups on a key no declaration can carry. Engine sends them with an undeclared operator on purpose, so serve them. See Handle Requests You Didn't Declare.
  • Refuse as early as you can tell. When the request's shape decides it (an unsupported filter or sort), refuse before any request to the external data source. When only the external data source knows (how many items match), make one bounded probe, such as a count, and refuse before reading data. A read budget that runs out refuses mid-read.
  • Write the message for the caller. It's the only explanation they get. Name the limit and what to change.
  • Never truncate instead. Returning fewer items than the limit tells Engine there are no more items.

The Art Institute of Chicago search API stops at its 1,000th result. The Art Institute connector from Map an External Data Source translates the filter and the sort first, so an undeclared one is refused before any request. It then counts when a read reaches past result 1,000, and rejects the read only if matches exist beyond it:

The Stripe connector throws QueryRejected when a filter plans more than 250 Stripe requests (a list request can still take several pages; top-level id constraints are retrieved instead and fall under the budget), when a read runs past its maxItemsPerQuery budget, and when a write reaches a data source with writes turned off. The fan-out bound and the writes switch are decided before any request. So is the budget for an id filter, whose retrieves spend it up front. A list read spends it as items arrive, so there the budget runs out during the read.

Don't use InvalidConfiguration or AccessDenied for these. Engine reports both as "this data source's configuration is invalid", which sends the caller looking in the wrong place.

Through Engine, a QueryRejected answers 422 with "The data source rejected the query" and your message nested inside:

response.json
{
  "message": "Query failed",
  "source": {
    "message": "The data source rejected the query",
    "source": {
      "message": "the extension returned an error: The Art Institute search API returns only the first 1000 results. Narrow the filter or read an earlier page."
    }
  }
}

When it works, the rejected request fails with your message. A shape-based refusal sends no request to the external data source, and a window refusal sends only its count request.

5. Validate Configuration in setup

Engine passes a data source's configuration to setup without checking it. Parse it there, once, and throw InvalidConfiguration with a message that names the key and what it expects. The Stripe connector's parser starts like this; the full parser is src/data-connectors/stripe/config.ts in the project that Build a Stripe Data Connector offers as a download:

src/data-connectors/stripe/config.ts
import type { StripeClient } from './client';
import { InvalidConfiguration } from '@monospace/extension-kit/data-connector';
import { isObject } from './client';
// …
/**
 * Nothing validates a connector's configuration before `setup` receives it, so this does. An
 * `InvalidConfiguration` from `setup` fails a connection test or a data source create with a 422.
 */
export function parseConfig(value: unknown): StripeConfig {
    if (!isObject(value)) throw new InvalidConfiguration('Stripe configuration must be an object.');

    const apiKey = value.apiKey;

    if (typeof apiKey !== 'string' || !/^(?:sk|rk)_(?:test|live)_\w+$/u.test(apiKey)) {
        throw new InvalidConfiguration('Stripe configuration `apiKey` must be a secret (`sk_`) or restricted (`rk_`) key.');
    }
  • InvalidConfiguration or AccessDenied from setup stops the data source until the extension is reloaded, because a retry can't fix the configuration. So does a permission denial there, raw or kept as cause. Another kit class, or a plain Error, thrown from setup fails the request with 422 "Invalid extension configuration", and the next request tries again. See Know What Deactivates a Data Source.
  • Connection tests start fresh. A test runs in a temporary connector, so after a test fails in setup, fixing the configuration and testing again works without a reload.
  • preflight can refuse a configuration too. It runs before setup to derive network permissions. When the configuration it needs is invalid (e.g., a base URL that isn't a URL), throw InvalidConfiguration there. That stops the data source the same way. Full validation still belongs in setup. See Derive Permissions in preflight.
  • Validate the format, then prove the credential in testConnection. setup can't tell a well-formed but wrong key from a real one without calling the external data source. The Stripe connector's testConnection makes one authenticated read, and its 401 becomes AccessDenied. Engine runs testConnection before it creates a data source or previews its schema, so a rejected credential fails both.
  • Throw from testConnection, don't answer { ok: false }. Engine reports { ok: false } as a failed connection with a fixed explanation, so your reason is lost. A thrown kit error carries your message.

AccessDenied thrown from testConnection or query, like the Stripe client's 401 mapping, fails only that request, reported as an invalid configuration, and leaves the data source running.

When it works, testing a data source with a malformed key fails with your message, and testing one with a well-formed but wrong key fails with the external data source's rejection.

6. Keep Secrets out of Messages

Only your error's code and message reach Engine; the cause (after the permission check), the stack and every other property are dropped. Engine maps the code to its own error, and the caller's HTTP response nests your message inside it. So the message is the one place a secret can leak, and the one place for context the caller needs.

Stripe's error bodies carry a request_log_url that contains the Stripe account id, and its 401 message echoes the key's last characters (sk_test_***********rong). The Stripe client keeps both out: toConnectorError in step 2 builds every message from Stripe's message field alone, or from the status and the path, and uses a fixed message for a rejected key.

  • Keep credentials out of messages. Never interpolate a key, token, or header into a message. When the external data source's own message echoes a credential, replace it, as the Stripe client does: "Stripe rejected the API key (HTTP 401)."
  • Keep the query string out of messages. It can hold the caller's filter values. The Stripe client's fallback message names the path only: "Stripe answered HTTP 500 to GET /v1/customers."
  • Never copy a response body into a message. Read only the fields you've reviewed. The Stripe client's readError reads Stripe's error type, code, param and message to classify the error, and only message reaches a message. The Art Institute connector names the API and the status, never the body.
  • Keep personal data out of messages, such as a customer's email from an item you read.

With these rules, the caller's 422 for a refused product delete carries Stripe's explanation, and no account id:

response.json
{
  "message": "Query failed",
  "source": {
    "message": "Foreign key constraint failed",
    "source": {
      "message": "the extension returned an error: This product cannot be deleted because it has one or more user-created prices."
    }
  }
}

When it works, triggering each mapped error over REST shows a response that contains nothing you wouldn't show the caller.

7. Check What Callers See

Test each mapping at two levels: the class your connector throws, and the response Engine returns.

In an artifact smoke, compare error.monospace.code, not instanceof. The built file bundles its own copy of the kit, so its classes aren't the ones your test imports. The kit's ErrorCode map names each code, so never hard-code a number. The Stripe smoke reads the code through a small helper, which answers PLAIN_ERROR for a plain Error:

scripts/smoke.mjs
import { ErrorCode } from '@monospace/extension-kit/data-connector';
// …
/** What a plain `Error` gives `codeOf`: it carries no kit code, and Engine reports it as a failed query. */
const PLAIN_ERROR = 'plain Error';

/** The kit error code the call throws, `PLAIN_ERROR`, or a failed assertion when it doesn't throw. */
async function codeOf(run) {
    try {
        await run();
    }
    catch (error) {
        assert.ok(error instanceof Error, 'the connector throws an Error');
        return error.monospace?.code ?? PLAIN_ERROR;
    }

    assert.fail('expected the connector to throw');
}

The refusal checks use it. This check reads the code directly, because it also asserts the message: a wrong key fails testConnection with AccessDenied and the fixed message, and the real key passes:

scripts/smoke.mjs
await check('a wrong key fails testConnection with AccessDenied, without echoing the key', async () => {
    const wrong = await descriptor.setup({ config: { apiKey: 'sk_test_wrongwrongwrong' } });
    const error = await wrong.testConnection().catch((thrown) => thrown);
    assert.equal(error.monospace?.code, ErrorCode.AccessDenied);
    assert.equal(error.message, 'Stripe rejected the API key (HTTP 401).');
    assert.deepEqual(await connector.testConnection(), { ok: true });
});

The unit tests cover what a live account can't produce on demand. They replace fetch with a fake Stripe and import the sources, so instanceof works there. These two check that a permission denial stays the cause, and that a server error's message names the path but not the query string or the log URL:

Then send the same failures through Engine. Send reads with Cache-Control: no-cache, so a cached success doesn't stand in for the failing call; see Bypass Caching. Some requests never reach your connector. Engine answers a caller's filter, sort, or operator the contract doesn't declare with a 422 of its own (codes 4003, 4012). A write to a collection without that operation gets 422 code 4010. A filter Engine adds, such as a permission rule's condition, isn't checked: it reaches your connector even when it compares a field or uses an operator the called operation doesn't declare. See Handle Requests You Didn't Declare.

For requests that reach your connector, a keyed miss (record: null) answers 404, and for an error the caller's HTTP status depends on the class: a constraint violation, a QueryRejected, a permission denial, InvalidConfiguration and AccessDenied answer 422. ConnectionFailed and RateLimited answer 500, as does a plain Error. RateLimited doesn't answer 429, and carries no Retry-After header. These are the statuses of errors your handlers throw. When setup deactivates the data source, the request that started it and every later one answer with that error's status: 422 "Invalid extension configuration" for InvalidConfiguration or AccessDenied. A connection test runs setup in a temporary connector, so a failure there fails only the test. A 503 "Data source is deactivated" while you test a new build can come from its query capabilities. With automatic reloading on, a build that changes them deactivates existing data sources until a restart or a workspace rebuild. See When the Capabilities Change and Know What the Caller Sees for the whole mapping.

When it works, every mapped class appears in your smoke with its code, and every Engine response you trigger carries a message the caller can act on.

See Test a Data Connector for the full test ladder.

See Also

MonospaceThe governed API layer for every app, person, and agent.

Copyright © 2026 Monospace Inc.