TMS: How to create Postgres -> BigQuery sync connection with Airbyte

You can get familiar with the GUI by reading this doc:How to make Airbyte connections (https://app.clickup.com/2496230/docs/2c5q6-69975/2c5q6-223735)

Database

Create a new airbyte_xyz123 database user if it does not exist yet

GCP Project

IAM

1. Create a new service account:
   1. Name: customer-airbyte-deployment_environment Example: soite-airbyte-stable
   2. Permissions:
      1. BigQuery Data Editor
      2. BigQuery Job User
2. Create a service account JSON key and save it somewhere. This will be used later in the Airbyte destination settings.

Big Query

Create the required datasets with Terraform:

Airbyte

Set up a source

1. Name: Customer Name - attracs-postge - tms_databasenamehere
2. Host: 34.88.30.27

This is the public IP address of the Postgres database

1. Port: 5432
2. DB Name: The name of the database you created the view tables for
3. Schemas: public
4. User: Give the username of your airbyte_xyz123 database user
5. Password: Give the password of your airbyte_xyz123 database user
6. JDBC URL Params: Keep default
7. Connect using SSL: Enable this
8. Replication Method: Keep default
9. SSH Tunnel Method: Keep default
10. Finally click Set up source.

Set up a destination

todo: add details like in source.

Set up a connection

1. Select the source you created
2. Select the destination you created
3. Change the settings:
   1. Destination Namespace: destination default
   2. Leave everything else default