Network access
What has to be open between your tools and Convalesce, in each direction, and where to allow it in AWS, Google Cloud and Azure.
Convalesce is a hosted service. It reaches your tools, and your tools reach it, over the internet. There are two directions to think about, and a tool only ever needs one of them.
| Kind of tool | Who connects | What you allow |
|---|---|---|
| One we read from: Postgres, Kafka, Snowflake, Databricks, Tableau and the like | Convalesce connects to your tool | Our address, inbound to the tool |
| One that sends to us: Airflow, Dagster, Prefect, Spark, Great Expectations | Your tool connects to Convalesce | Our endpoint, outbound from the tool |
Each tool's own page has a Network section with what applies to it. This page is the whole picture.
Tools we read from
Every read comes from one fixed address:
34.66.85.47Allow it wherever inbound connections to the tool are limited. Use the single address, not a range.
| Where the tool runs | Where to allow the address |
|---|---|
| AWS | The security group of the instance, cluster or load balancer. For RDS, switch on the instance's public access as well. |
| Google Cloud | A VPC firewall rule. For Cloud SQL, an authorised network. |
| Azure | A network security group rule, or the service's own firewall rules, as on Azure Database for PostgreSQL. |
| Your own servers | The firewall in front of the service, and for Postgres its pg_hba.conf. |
The tool needs a public address
Give each tool we read from an address that is reachable from the internet, and limit who may use it to the one address above. That is the whole of the setup: there is no agent to install and no tunnel to keep running.
Tools read through a cloud's own API
S3, Glue and DynamoDB are read through AWS's public API. Snowflake, Databricks and Tableau Cloud have public addresses of their own. There is no firewall rule to add for these.
The exception is a restriction you have added yourself. Each of these has to include the address above:
- an S3 bucket policy or IAM condition on
aws:SourceIp - a Snowflake network policy
- a Databricks workspace IP access list
Whose credentials are used
Convalesce reads only with a key, a token or a database user that you create for it in your own account, and that you can revoke at any time.
Tools that send to us
A tool that sends to us calls one endpoint over HTTPS:
https://api.convalesce.io/openapi (port 443)Nothing has to be opened inbound to your tool. The call out has to be allowed from wherever the tool runs.
| Where the tool runs | What it needs |
|---|---|
| A machine or container with normal internet access | Nothing more. |
| Amazon MWAA with private routing | A route out through a NAT gateway for the environment's subnets. Reading the key from Secrets Manager needs that route too, or a VPC endpoint. |
| Cloud Composer with Private IP | Cloud NAT on the environment's network. |
| EMR in a private subnet | A route out through a NAT gateway. |
| An AWS Glue job with a VPC connection | A NAT gateway in that VPC. A job with no connection already has a route out. |
| Databricks serverless | The endpoint's host allowed in the serverless network policy, when one restricts outbound access. |
| Kubernetes | The endpoint allowed by any NetworkPolicy or egress gateway that limits outbound traffic. |
If your network only lets named hosts out, the host to allow is
api.convalesce.io.
How to tell a network problem from a wrong credential
- A tool we read from: press Test the connection. A timeout or "could not reach" means the address is not allowed or not reachable. A refusal that names a user or a permission means the network is fine and the credential or grant is not.
- A tool that sends to us: run
convalesce-emit checkwhere the tool runs, or look in its log for lines startingconvalesce:. A connection error means the call out is blocked. A401or403means it got through and the key is wrong.