Manage Data Sources

Overview

A data connector extension does nothing in a workspace until someone creates a data source from it. A data source is a data connector configured in one workspace, with its own configuration and granted network permissions. It points at the external data source, the API, service, or database the connector reads and writes, and contributes its collections to the workspace.

Every installed extension is available to every workspace on the instance (your running Monospace deployment). This page covers what happens after you install a data connector extension: finding its connectors, creating data sources, and keeping them working when the extension changes. See How Data Connectors Run for what Engine does each time a data source starts.

List Installed Data Connectors

List what installed extensions provide with GET /api/{workspace}/connectors/data. It lists data connector extensions only, not the built-in database connectors. It requires the extension:read entitlement in the workspace. The SDK has no method for connectors, so call the endpoint directly:

curl -g "https://example.monospace.io/api/blog/connectors/data" \
  -H "Authorization: Bearer YOUR_API_KEY"

const response = await fetch('https://example.monospace.io/api/blog/connectors/data', {
  headers: {
    Authorization: 'Bearer YOUR_API_KEY',
  },
});

const { data } = await response.json();

Each entry's id is the entry id a data source names as its provider. extensionId is the extension it belongs to, and permissions lists the hosts it declares:

{
  "data": [
    {
      "id": "example/artic-connector",
      "version": "0.1.0",
      "name": "Art Institute of Chicago",
      "icon": "lucide:landmark",
      "extensionId": "example/artic",
      "permissions": [
        {
          "permission": "net:api.artic.edu",
          "reason": "Read artworks and artists from the Art Institute of Chicago API"
        }
      ]
    }
  ]
}

Add a Data Source in Studio

Studio offers installed data connectors when you add a data source, each marked Extension. These steps add the Art Institute of Chicago connector from the Data Connector Quickstart:

1. Open Data Model and choose Add Source.
2. Choose Art Institute of Chicago.
3. Enter artic as the Name, and leave Connection Parameters at {}. This connector has no configuration.
4. Review Permissions: the hosts the connector declares, each with its reason.
5. Choose Test Connection, then Continue.
6. Under Select Collections to Add, keep artworks and artists selected. Choose Review, then Add 2 Collections.

Studio grants the permissions the connector declares. It asks you to approve permissions only when the connector needs more than it declares, which this one never does.

Creating a data source, in Studio or over the API, runs the connector's testConnection first when the connector has one. A test that fails stops the creation.

Approve Network Permissions

A data connector reaches only the hosts its data source was granted. Whoever creates the data source approves the hosts the connector requires: the net: permissions in its manifest, plus any its preflight derives from the configuration.

See Approve Permissions in Network Permissions for the approval flow over the API.

Update Data Sources After an Extension Update

When you update an extension with automatic reloading on, Engine restarts every data source whose entry id is still in the new build. A data source whose entry the new build removes or renames isn't restarted. Without automatic reloading, restart the instance. What else you do depends on what changed:

A reload swaps code, not the schema a data source was created with. Until you refresh the schema, Engine keeps accepting requests based on the old one. Requests the new code can't serve fail. See Import Changes for the introspect endpoint.

Refreshing the schema keeps the data source's configuration and granted permissions. That's why a change in required hosts needs a new data source. A change to a permission's reason alone needs nothing, because Engine doesn't compare reasons.

Recreate a Data Source

Delete the data source, then create it again with the new permissions or configuration. Recreating isn't an update in place: deleting a data source deletes the workspace's stored definitions of its collections and of the relations between them. The new data source gets a new id and a freshly imported schema, with the relations its connector declares.

In Studio, remove it from Data Model and add it again. Over the API, you need its id. List the data sources in a workspace with their providers with GET /api/{workspace}/schema/structure/sources. The fields parameter selects the fields each data source returns; a request without it is refused. The SDK has no method for data sources, so call the endpoint directly:

curl -g "https://example.monospace.io/api/blog/schema/structure/sources?fields=id,apiName,provider" \
  -H "Authorization: Bearer YOUR_API_KEY"

const params = new URLSearchParams({
  fields: 'id,apiName,provider',
});

const response = await fetch(
  `https://example.monospace.io/api/blog/schema/structure/sources?${params}`,
  {
    headers: {
      Authorization: 'Bearer YOUR_API_KEY',
    },
  },
);

const { data } = await response.json();

The list is paged like any other: without limit, it returns at most the instance's default page size (100 items unless configured). The list includes the workspace's own _system data source, which can't be deleted:

{
  "data": [
    {
      "id": "00000000-0000-0000-0000-000000000000",
      "apiName": "_system",
      "provider": "postgres"
    },
    {
      "id": "f353377a-60dc-4edd-8052-faeb9bdfe53f",
      "apiName": "artic",
      "provider": "example/artic-connector"
    }
  ]
}

To find one data source by name, add a filter, e.g. filter[apiName][_eq]=artic.

Delete the data source, then create it again as described in Network Permissions.

Delete a Data Source

Delete a data source by id with DELETE /api/{workspace}/sources/data/{source}. In Studio, open Data Model, open the menu next to the data source, choose Remove Data Source, and confirm with Remove. Deleting requires the dataSource:delete:{id} entitlement for that data source.

Before You Remove an Extension

Delete the data sources that use the extension's data connectors first, then remove the extension.

A data source whose extension is removed keeps serving on the code it already loaded. It stops at its next restart, because the connector file is gone.

See Also

• Install and Manage Extensions: place, reload, update, and remove extension files
• Data Connector Quickstart: build a connector and create its first data source in Studio
• How Data Connectors Run: the data source lifecycle, restarts, and failures
• Network Permissions: declare and approve the hosts a connector calls
• Limitations: what you can't change on an existing data source