Databricks

Step 1 of 11: Name it

What to call this connection, and the deployment it belongs to.

Step 2 of 11: Name the workspace

Workspace URL, warehouse and catalog.

Find the workspace URL, such as https://dbc-a1b2c3d4-e5f6.cloud.databricks.com, and the ID of the SQL warehouse the metadata queries run on (SQL Warehouses → your warehouse → Overview).

Without a warehouse, lineage and usage fall back to the REST API, and tags and the legacy hive_metastore catalog are skipped.

There is no network rule to add: Convalesce calls the workspace's own address, which is public. The exception is a workspace with IP access lists switched on: it has to allow 34.66.85.47, the address Convalesce connects from.

Step 3 of 11: Choose how to sign in

A service principal or a personal access token.

A service principal belongs to no person and can be revoked on its own. A personal access token acts as whoever created it, so prefer the service principal outside a trial.

Step 4 of 11: Add the service principal

Its application ID and an OAuth secret.

Create a service principal in Settings → Identity and access → Service principals, and generate an OAuth secret for it. Note its application ID, which is the OAuth client ID.

Step 5 of 11: Grant the read access

Catalog access and the warehouse.

Run in a notebook or the SQL editor, as the catalog's owner or a metastore admin.

Then give the same principal CAN USE on the warehouse, in SQL Warehouses → your warehouse → Permissions. That is the least it needs to run the metadata queries; it cannot start, stop or edit the warehouse.

With a personal access token, replace the principal with the email of the user who owns the token.

-- Read the catalog's metadata. BROWSE shows its tables, views and models
-- without the right to read a row.
GRANT USE CATALOG ON CATALOG main TO `6f1c-2a3b`;
GRANT USE SCHEMA ON CATALOG main TO `6f1c-2a3b`;
GRANT BROWSE ON CATALOG main TO `6f1c-2a3b`;

-- SELECT is what the connector reads table details and profiles with.
-- Granted on the catalog, so tables created later are covered.
GRANT SELECT ON CATALOG main TO `6f1c-2a3b`;

-- Read the catalog's metadata. BROWSE shows its tables, views and models
-- without the right to read a row.
GRANT USE CATALOG ON CATALOG sales TO `6f1c-2a3b`;
GRANT USE SCHEMA ON CATALOG sales TO `6f1c-2a3b`;
GRANT BROWSE ON CATALOG sales TO `6f1c-2a3b`;

-- SELECT is what the connector reads table details and profiles with.
-- Granted on the catalog, so tables created later are covered.
GRANT SELECT ON CATALOG sales TO `6f1c-2a3b`;

Step 6 of 11: Grant lineage and usage (optional)

Optional: the system tables.

Lineage comes from system.access and usage from system.query, both read through the SQL warehouse.

Only a metastore admin or an account admin can run these. Anyone else gets PERMISSION_DENIED: User does not have MANAGE on Catalog 'system'. Many workspaces already let every user read these tables; the connection test shows whether lineage is readable.

If your security policy does not allow these, everything else still works; lineage and usage then come from the REST API, which is slower and sees less.

-- Lineage: system.access
GRANT USE CATALOG ON CATALOG system TO `6f1c-2a3b`;
GRANT USE SCHEMA ON SCHEMA system.access TO `6f1c-2a3b`;
GRANT SELECT ON TABLE system.access.table_lineage TO `6f1c-2a3b`;
GRANT SELECT ON TABLE system.access.column_lineage TO `6f1c-2a3b`;

-- Usage: system.query
GRANT USE SCHEMA ON SCHEMA system.query TO `6f1c-2a3b`;
GRANT SELECT ON TABLE system.query.history TO `6f1c-2a3b`;

Step 7 of 11: Choose what to read

Lineage, usage, notebooks, models.

Blank keeps the default shown in each hint.

Leave the legacy hive_metastore catalog off on a Unity Catalog workspace; reading it needs USAGE and READ_METADATA there.

Step 8 of 11: Choose what is read (optional)

Optional: narrow it to some tables.

Everything the credential can see is read unless you narrow it here. List the tables you want, the ones to leave out, or both.

Step 9 of 11: Test the connection

Check Convalesce can reach it with what you entered.

The test runs on the same worker a real run would, with the recipe exactly as it will be saved, so it fails the way a run would.

Step 10 of 11: Choose how often

How often Convalesce reads it.

Step 11 of 11: Review and connect

Check everything, then save the connection.