Data Connectors
@monospace/cli 0.2 and @monospace/extension-kit 0.3. Overview
A data connector extension teaches Monospace to work with a system it has no built-in connector for: a SaaS API, an internal service, or a database without a native connector. These docs call that system the external data source.
Think of a data connector as a translator. Callers ask Monospace for the items (single data objects) in collections of a workspace, through the REST API, the SDK, or Studio. Each item carries named values called fields. Monospace hands each question to your connector in a structured form. Your connector answers it by calling the external data source, and Monospace returns the result like any other collection's.
You make a connector available to a workspace by creating a data source: the connector plus its configuration (e.g., an API key) and the hosts it's allowed to reach.
Live Queries, Not Sync
A data connector doesn't import anything. Monospace never syncs or replicates external data. Query results are cached like any data source's. Engine translates each request the cache doesn't answer into calls to the connector, and the connector makes the requests to the external data source that each call needs.
| Consequence | What it means for you |
|---|---|
| Freshness | A cached result is served until it expires (5 minutes by default) or a write through Monospace invalidates it. Changes made directly in the external data source appear after the cache entry expires, or immediately with a cache bypass. See Caching Limitations for the edge cases |
| Load on the external data source | Uncached reads cost requests to it. A total count can add a separate count read, and including a to-many relation, which returns several related items, adds one read per parent item by default |
| Rate limits | Monospace traffic counts against the external data source's rate limits and quotas. Engine doesn't retry: when it throttles and the connector doesn't retry, the request fails |
| Availability | When the external data source is down, requests that need it fail unless the cache answers them |
| Query power | Callers can ask a collection only for what the connector declares. The connector can answer more than the external data source supports natively (e.g., by filtering fetched items itself), at the cost of extra reads |
How a Request Flows
A read against a connector collection passes through Engine before it reaches your code.
Engine Checks the Request
Engine authenticates the caller, applies their permissions, and checks the request against the collection's declared operations. A request outside the declaration fails with a validation error before the connector sees it. A few caller operators are derived rather than declared (e.g., _neq arrives as not around equals, and Engine offers it only where the filter declares equals and not). See Map Caller Operators to the Wire.
Engine Sends an Operation
Each call to the connector carries one operation (e.g., readMany with a filter, sort, limit, and offset). One caller request can produce several calls. Engine adds shapes of its own: a default limit (100 unless the instance, your Monospace deployment, changes it), the caller's permission rules as filters, count reads, and extra reads to resolve relations. Several filtered permission rules arrive joined by or, and a field-level update rule can arrive wrapped in not, even when the collection declares neither. Engine doesn't check the filters it sends against the called operation's declaration, so a permission rule's condition can also compare a field the operation doesn't declare. The connector refuses what it can't answer. See Handle Requests You Didn't Declare.
Connector Calls the External Data Source
The connector's query handler maps the operation to requests to the external data source, pages through its results as needed, and returns exactly the matching items. An operation on one item by its key answers { record: <item> }, or { record: null } when there's none. Each item carries every field the operation selects, with an explicit null for a missing value; an item without a selected field fails the request. Engine doesn't check the returned items against the filter or the sort, so the connector alone is responsible for returning exactly the items the operation asks for.
Engine Answers the Caller
Engine combines the connector's items with related data, shapes the response, and returns it.
What You Build
A data connector extension has two parts you write: an extension config and a connector module. The module can import other source files of yours; the build bundles them. The examples below come from the Data Connector Quickstart, which reads the Art Institute of Chicago API.
The extension config names the extension and its one entry, and declares the one host the connector may reach:
{
"$schema": "./node_modules/@monospace/cli/schemas/extension.config.schema.json",
"entries": [
{
"engine": {
"entrypoint": "./src/data-connectors/artic/index.ts",
"permissions": [
{ "permission": "net:api.artic.edu", "reason": "Read artworks and artists from the Art Institute of Chicago API" }
]
},
"icon": "lucide:landmark",
"id": "example/artic-connector",
"name": "Art Institute of Chicago",
"type": "dataConnector",
"version": "0.1.0"
}
],
"icon": "lucide:landmark",
"id": "example/artic",
"name": "Art Institute of Chicago",
"version": "0.1.0"
}
The entry's code is the connector module. Its default export is the connector. It imports the schema, the function that calls the Art Institute API, and the read handlers from sibling files, which the Data Connector Quickstart builds one by one:
import { defineDataConnector } from '@monospace/extension-kit/data-connector';
import { search } from './api';
import { readMany, readOne } from './read';
import { schema } from './schema';
const COLLECTIONS = new Set<string>(schema.collections.map((collection) => collection.name));
export default defineDataConnector({
setup() {
return {
introspect() {
return schema.toSchemaDocument();
},
async query({ query }) {
// Monospace sends only declared collections, and this connector declares no namespaces.
// For any other collection, throw a plain `Error`.
if (query.namespace !== undefined || !COLLECTIONS.has(query.collection)) {
throw new Error(`Unknown collection \`${query.collection}\`.`);
}
switch (query.op) {
case 'readMany':
return await readMany(query);
case 'readOne':
return await readOne(query);
default:
// The schema declares no writes, so Monospace offers callers none.
throw new Error('The Art Institute of Chicago API is read-only.');
}
},
async testConnection() {
await search('artworks', { size: 0 });
return { ok: true };
},
};
},
});
| Part | Role |
|---|---|
setup | Runs when a data source starts. Receives the data source's configuration and returns the handlers |
introspect | Returns the schema: the collections, their fields, and the operations each collection accepts |
query | Receives one operation per call (e.g., readMany, readOne) and answers it by calling the external data source. Calls can run concurrently |
testConnection | Optional. Checks that the external data source is reachable. Engine runs it on a connection test, and before it creates a data source or previews its schema. Without it, a connection test fails as unsupported |
permissions | The hosts this data connector may reach. Any other network request from its code fails |
The monospace CLI builds these into a manifest and one self-contained JavaScript file, and you place the output in your instance's install location. You then create a data source (the connector configured in a workspace), approve its network permissions, and query its collections like any other. The Data Connector Quickstart walks through each step.
Operations Contract
Each collection a connector exposes declares its operations: which of readMany, readOne, create, updateMany, updateOne, deleteMany, and deleteOne it serves, and which filters, sorts, and fields each one accepts.
The declaration is a promise that gates callers. Engine builds the API schema, the OpenAPI document, and Studio's controls from it.
| Declaration | Effect |
|---|---|
| You declare less than the external data source supports | Studio hides the control, and the API rejects the request. Engine still needs some declarations for its own calls: an update on a collection with a stable ID needs readMany to carry its read-backs, a connector without query capabilities needs more for its writes, and a relation needs the target to declare a filter on its key (in joined by and, or equals joined by and and or), unless the key's type can't declare one. Without them, data source creation fails, or the relation isn't offered and Engine logs "The engine offers less than the collection declares." |
| You declare exactly what the external data source supports | Callers can still send requests the contract can't restrict, such as an unfiltered read or a large limit. An instance-wide maximum limit, when set, bounds only a limit the caller sends. The default limit isn't checked against it, and Engine's own relation reads can arrive without limit. The connector refuses or pages through those at runtime |
| You declare more than the external data source supports | Callers send requests the connector must refuse at runtime, or answer by scanning external data |
Declare only what the external data source answers efficiently and correctly. See Operations and Operators for the declaration format and the rules Engine enforces, including Rules for Engine's Own Calls.
Fit Check
Answer these questions about your external data source before you build. Each "yes" points to a limitation to design around.
| Question | If yes |
|---|---|
| Can some collections only be listed when scoped by a parent or a filter (e.g., a channel, a customer, a date range)? | A connector can't declare a required filter. It refuses unscoped reads at runtime. See Required Filters |
| Is paging cursor-only, or does the external data source stop at a maximum depth? | Callers page by limit and offset. On a cursor-only source the connector walks pages and discards items up to offset. Past a maximum depth it refuses. See Paging |
| Do callers need aggregates, grouping, full-text search, or vector search? | The query API has none of these yet; its only aggregate is a total count. Keep up with what's coming on our Roadmap. See Query Shapes |
| Are items keyed by more than one field? | Single-item operations need a single-field primary key. See Keys |
| Do references point at things that may not exist in the external data source (e.g., external phone numbers, deleted objects)? | Engine doesn't check that a _connect target exists when the key sits on the written item, so the external data source decides whether to accept it. See Relations |
| Does a write to the external data source answer without the written object? | Leave that write's query capability undeclared. Engine then makes extra calls by stable ID: it asks create for the stable ID only and reads the created items back, and it sends updates and deletes through updateMany and deleteMany. create must still answer with each created item's stable ID. A collection with such a write needs a primary or unique index and declarations that carry those calls. See Declare Query Capabilities |
| Must several writes succeed or fail together? | Engine runs no transaction across connector writes, so a failure partway leaves earlier writes applied. See Writes |
| Will credentials, hosts, or permissions change after the data source is created? | Configuration and permissions are fixed at creation. Once a connector version that changes its required permissions is loaded, every request that reaches an existing data source's connector fails. Cached results are still served. We're working on letting you change a data source's configuration after it's created. See Configuration and Network |
| Does the external data source's schema change often? | Data sources keep the schema from creation until you reintrospect. See Lifecycle |
A few "yes" answers are normal for SaaS APIs. Many "yes" answers on the collections you need most mean the external data source is a poor fit.
Built-In Database Connectors
If the external data source is a PostgreSQL, Supabase, MySQL, or MariaDB database, use a built-in database connector instead of an extension. Built-in connectors introspect the database directly and support capabilities data connector extensions don't have, including filters across relations and transactions.
See Connectors for the supported databases.
Next Steps
More example connectors are available on request.
- Data Connector Quickstart: build a read-only connector for the Art Institute of Chicago API, with
artworks,artists, and a relation between them - Build a Stripe Data Connector: build a Stripe test-mode connector with
customers,products, andprices, including writes - Map an External Data Source: translate filters, paging, and keys onto a real external data source
- Manage Data Sources: create, update, and delete the data sources that use a connector
- Data Connector Limitations: every current limitation, with workarounds