How to re-sync fresh Airbyte data in case of e.g. corrupted or duplicate data

BigQuery

You can get fresh data from Airbyte by deleting all of the dataset tables. For now the easiest way is to use gcloud command-line program to delete all of the tables. For this purpose we have a script that loops through all of the tables of a given dataset and outputs delete commands for each table. You can automate this script to also run the list of delete commands but we feel it's safer to err on the side of caution and manually run the list of delete commands after confirming it is what we want to delete.