Data Connectors

Data Connector Runtime Limits

Hard numbers and fixed rules for data connector extensions at runtime, and the error or behavior each one produces.

Preview

Extensions are in preview and can change in breaking ways between releases. Use them in non-production environments only. See Stability. Written for Monospace 1.0.0, @monospace/cli 0.2 and @monospace/extension-kit 0.3.

Overview

Every data connector extension runs under the limits on this page, in each data source: the connector configured in a workspace. A data source exposes collections of items, each a single data object, and runs on the instance, your Monospace deployment. None of the limits are configurable per extension. The page lists each limit with the behavior you see when you hit it.

Limit	Value
Module evaluation timeout	30 seconds
`setup` timeout	30 seconds
Liveness check	Ping after 30 seconds of silence; unresponsive after another 30
Per-request timeout	None
Default page size	100 items, from `MONOSPACE_QUERY_LIMIT_DEFAULT`
Maximum page size	Unlimited, unless `MONOSPACE_QUERY_LIMIT_MAX` is set
Bundle shape	One JavaScript file per entry, no imports
Isolates	One per data source, serving requests concurrently

Module Evaluation and `setup`

Your instance gives each start of a data source two 30-second windows. Neither can be changed.

Phase	Window	When exceeded
Evaluating the entry file, including top-level `await`	30 s	The start fails with `the extension did not finish loading within 30s`
`setup`, from call to returned handlers	30 s	The start fails with `the entrypoint stopped answering`

Neither failure deactivates the data source. The request that started it fails. Requests that arrive while it starts don't wait: they fail at once with 503 "Data source is restarting". The next request starts it again from scratch. Keep network calls out of module scope, and keep setup to building clients and validating configuration.

preflight runs between the two windows and has no window of its own. It must return synchronously. A preflight that blocks the thread falls under the liveness check.

Liveness Check

Once a data source is serving, the instance watches its worker for silence:

After 30 seconds without any message from the worker, the instance sends a ping.
The worker's runtime answers the ping itself, even while handlers are waiting on the network.
If another 30 seconds pass without any message, the instance marks the worker unresponsive.

Messages here are the worker's protocol messages, such as answers and ping replies. console output goes to the log separately and doesn't reset the silence timer.

An unresponsive worker stops serving, and the instance replaces it on a later request:

Requests in flight fail and mark the worker for replacement. With none in flight, the first request sent to the stopped worker fails and marks it instead.
The next request starts a new worker and runs setup again. The old worker ends when the new one takes over.
Failed requests aren't replayed. The caller decides whether retrying a failed write is safe.

The worker misses a ping only when its JavaScript thread is blocked. A synchronous loop or a long CPU-bound computation blocks every request to the data source. The instance fails those requests 30 to 60 seconds after the thread blocks, depending on when the worker last sent a message. It then replaces the worker as above. Keep synchronous work in handlers short.

Per-Request Deadline

The instance sets no deadline on query, introspect or testConnection. A handler that awaits a slow external data source waits as long as that source takes, and so does the caller's request. A client can stop waiting sooner (Studio's connection test gives up after 60 seconds), but nothing cancels the handler: a write it started can still complete. Put your own deadline on outgoing calls: pass AbortSignal.timeout(ms) as the signal of each fetch, and read the body inside the same try. The signal also aborts reading the body. When the deadline passes, fetch rejects with an error named TimeoutError. A deadline set this way bounds one request, not the operation: a read that pages through the external data source sends several requests, each with a fresh deadline. See HTTP Through fetch for an example.

Page Size

For a caller's read of a collection, the readMany operation arrives with the page size the caller asked for, or the default:

Caller sends	Connector receives
No `limit`	`limit: 100`, or the value of `MONOSPACE_QUERY_LIMIT_DEFAULT`. No `limit` when `config.yaml` sets `query_limit_default: null`
`limit=25`	`limit: 25`. Callers can set `limit` only when `readMany` declares it
`limit=-1` or `limit=0`	No `limit` when `MONOSPACE_QUERY_LIMIT_MAX` is unset. Return every matching item. When the maximum is set, `limit` is the maximum
`limit` above `MONOSPACE_QUERY_LIMIT_MAX`	Nothing; the instance rejects the request with `Cannot set "limit" above the max`
`limit` below `-1`, or a negative `offset`	Nothing; the instance rejects the request. `limit` is an integer from `-1` up, and `offset` a non-negative integer

The default isn't checked against the maximum. With a default of 100 and a maximum of 25, a read without limit still arrives with limit: 100. A configured default of 0 isn't a caller's limit=0: reads without limit then arrive with limit: 0 and return no items. Reads the instance issues on its own can carry other limits. A to-many relation include follows the same rules for its own limit. It arrives as one batched read only when it resolves to no limit and no offset, and otherwise as one read per distinct, non-null parent key, sent one after another; see Relation Reads for every case.

A connector receives the default limit even when its readMany declares no limit. Count requests are different: they carry limit: 0 and count: true, and expect totalCount with no items. See Count Reads.

A connector declares no ceiling of its own. When an external data source caps its page size, the connector must page internally until it fills limit, because a short page reads as the last page.

A read without limit isn't streamed. The connector answers with every matching item in one records array, and the instance holds the whole answer in memory. Bound it with a fetch budget in the connector, and throw when the budget runs out: the instance reads a partial answer as the complete result.

One File, No Imports

The instance loads exactly one file per entry: the manifest's engine.entrypoint. Its module loader refuses every other module.

In the entry file	Result
Static `import`, `export ... from`	The file fails to load with `an extension cannot import <specifier> from <referrer>`
Dynamic `import()`	The returned promise rejects with the same message when the call runs
`require()`	Throws at runtime; `require` isn't defined
No default export, or a default export without a `setup` function	The file fails to load with `the entrypoint's default export must be a descriptor with a \`setup` function`

A file that fails to load deactivates its data sources until the extension is reloaded. monospace extension build checks each bundle for imports and the default export before it writes that bundle. It doesn't run the file, so a default export without a setup function passes the build and fails at load. A failed build without --watch leaves the previous output in place. See CLI.

Available Globals

The worker is a Deno isolate with web platform APIs and no Node layer.

Available	Not available	Result when used
`fetch`, `URL`, `crypto`, `TextEncoder`, `TextDecoder`, `ReadableStream`, `AbortController`, `AbortSignal.timeout`, `setTimeout`, `structuredClone`, `console`	`node:*` modules, `require`, npm packages at runtime	The file fails to load, or `require` is undefined
	File reads and writes (`Deno.readTextFile`, `Deno.writeTextFile`)	Throws `NotCapable`
	Environment variables (`Deno.env`)	Throws `NotCapable`
	Subprocesses (`Deno.Command`)	Throws `NotCapable`
	`Worker`	Throws; web workers are disabled
	stdin, stdout, stderr (`Deno.stdout`, `Deno.stderr`)	Reads get nothing; writes are discarded
	Hosts outside the data source's grant	`fetch` rejects with `NotCapable`. Keep it as the `cause` when you wrap it. See Handle a Denied Request

console output goes to the instance log at info level, tagged with the data source and provider. Unbounded recursion throws a catchable RangeError instead of crashing the worker.

Deno.exit() ends only the worker, not the instance. The instance replaces the worker as described in Liveness Check.

Concurrency

Each data source gets its own isolate on its own thread. Two data sources that use the same connector share no memory.

Inside one isolate, requests run concurrently. The instance dispatches each request as soon as it arrives, without waiting for earlier ones to finish. Treat everything you create in setup as shared between in-flight requests:

Don't keep per-request state in variables created by setup.
Guard shared caches and token refreshes against concurrent callers.
Limit concurrent calls yourself when the external data source rate-limits. The instance puts no cap on how many requests run at once, and a per-request deadline bounds how long each call waits, not how many run. Engine runs the per-parent reads of one included relation up to 16 at a time, and concurrent caller requests each get their own 16.

State in the isolate lasts until the worker restarts. A restart follows an extension reload, an unresponsive worker, a crash or Deno.exit(), and runs setup again.

User Agent

Requests from fetch carry the default user agent Monospace/0.1.0. The version is the runtime component's, not your instance's release.

A User-Agent header you set on a fetch call replaces the default. Set it when the external data source asks clients to identify themselves.