Common Airbyte Connection Examples

Suggest Edits

Running the following commands will import data from the given external source into your Faros application. To understand more about how these commands are created and to connect to other sources check out Importing Data in Hybrid Deployment Mode doc.

📘

Prerequisites

Download our Airbyte Local CLI script and make sure you have the required bash, docker, jq, and tee installed.

Bitbucket

./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "bitbucket-feed" \
  --src.feed_cfg.workspaces '["my-bitbucket-workspace-id-1", ...]' \
  --src.feed_cfg.token $BITBUCKET_TOKEN \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mybitbucketsrc"

Buildkite

./airbyte-local.sh \   
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "buildkite-feed" \
  --src.feed_cfg.organization "my-buildkite-org" \
  --src.feed_cfg.token $BUILDKITE_TOKEN \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mybuildkitesrc"

Changeset

./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "changeset-feed" \
  --src.feed_cfg.faros_api_url "https://prod.api.faros.ai" \
  --src.feed_cfg.faros_api_key $FAROS_API_KEY \
  --src.feed_cfg.graph_name "default" \
  --src.feed_cfg.github_cfg.auth.kind "token" \
  --src.feed_cfg.github_cfg.auth.token $GITHUB_TOKEN \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mychangesetsrc"

CircleCI

./airbyte-local.sh \
  --src farosai/airbyte-circleci-source:latest \
  --src.token $CIRCLECI_API_TOKEN \
  --src.project_names '["vcs-slug/org-name/repo-name-1", ...]' \
  --src.cutoff_days 90 \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mycirclecisrc"

GitHub

./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "github-feed" \
  --src.feed_cfg.auth_cfg.auth "token" \
  --src.feed_cfg.auth_cfg.personal_access_token $GITHUB_TOKEN \
  --src.feed_cfg.org_repo_list '["my-github-org-1/*", "my-github-org-2/repo-1", ...]'  \ 
  --src.feed_cfg.repos_query_mode.faros_api_url "https://prod.api.faros.ai" \
  --src.feed_cfg.repos_query_mode.faros_api_key $FAROS_API_KEY \
  --src.feed_cfg.repos_query_mode.query_mode "FarosGraph" \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mygithubsrc"

GitLab

./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "gitlab-feed" \
  --src.feed_cfg.token $GITLAB_TOKEN \
  --src.feed_cfg.repos_query_mode.faros_api_url "https://prod.api.faros.ai" \
  --src.feed_cfg.repos_query_mode.faros_api_key $FAROS_API_KEY \
  --src.feed_cfg.repos_query_mode.query_mode "FarosGraph" \
  --src.feed_cfg.repos_query_mode.group_names '["my-gitlab-group-1", ...]' \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mygitlabsrc"

Jenkins

./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "jenkins-feed" \
  --src.feed_cfg.server_url $JENKINS_SERVER_URL \ 
  --src.feed_cfg.user $JENKINS_USER \
  --src.feed_cfg.token $JENKINS_TOKEN \
  --src.feed_cfg.jobs_query_mode.query_mode "All" \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "myjenkinssrc"

Jira

./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "jira-feed" \
  --src.feed_cfg.domain $JIRA_SERVER_URL \
  --src.feed_cfg.personal_access_token $JIRA_TOKEN \
  --src.feed_cfg.projects_query_mode.faros_api_url "https://prod.api.faros.ai" \
  --src.feed_cfg.projects_query_mode.faros_api_key $FAROS_API_KEY \
  --src.feed_cfg.projects_query_mode.query_mode "FarosGraph" \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "myjirasrc"

Additional Resources

Create a GitHub App to Authenticate Faros to Multiple Orgs

You can create a GitHub App and install it on your multiple GitHub organizations to easily enable read-only access from the Faros connector to your desired organizations and repositories.

  1. Navigate to either your profile settings page or your organization's settings page and create a new GitHub app.
  • At the bottom of the creation page, select "Any account" to allow your app to be installed any user or organization.
  • The Webhook option is not required.
  • The app requires the following read-only permissions:
    • Repository permissions: Administration, Contents, Issues, Metadata, Projects, Pull Requests
    • Organization permissions: Members
  1. On your app's settings page, create a private key and a client secret. These values, along with the App Id and Client ID, are required by the Faros connector.
  2. Install the GitHub app on your desired organizations. You can either grant access to all repositories in an org, or only grant access to specific repositories per org.
  3. Run the Faros GitHub connector:
./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "github-feed" \
  --src.feed_cfg.auth_cfg.auth "app" \
  --src.feed_cfg.auth_cfg.app_id $GITHUB_APP_ID \
  --src.feed_cfg.auth_cfg.private_key $GITHUB_APP_PRIVATE_KEY \
  --src.feed_cfg.auth_cfg.app_cfg.auth "client" \
  --src.feed_cfg.auth_cfg.app_cfg.client_secret $GITHUB_APP_CLIENT_SECRET \
  --src.feed_cfg.auth_cfg.app_cfg.client_id $GITHUB_APP_CLIENT_ID \
  --src.feed_cfg.org_repo_list '["my-github-org-1/*", "my-github-org-2/repo-1", ...]' \
  --src.feed_cfg.repos_query_mode.faros_api_url "https://prod.api.faros.ai" \
  --src.feed_cfg.repos_query_mode.faros_api_key $FAROS_API_KEY \
  --src.feed_cfg.repos_query_mode.query_mode "FarosGraph"
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mygithubsrc"

Create a GitLab App to Authenticate Faros

You can create a GitLab App and install it to easily enable read-only access from the Faros connector to your desired organizations and repositories.

  1. In the top-right corner of your GitLab account, select your avatar.
  2. Select Edit profile.
  3. On the left sidebar, select Access Tokens.
  4. Enter a name and set the expiry date to "no expiration" for the token.
    • The app only requires the following read-only permission:read_api
  5. Select Create personal access token.
  6. Run the Faros GitLab connector:
./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "gitlab-feed" \
  --src.feed_cfg.token $GITLAB_TOKEN \
  --src.feed_cfg.repos_query_mode.group_name "mygroup" \
  --src.feed_cfg.repos_query_mode.repositories '["repo1", "repo2"]' \
  --src.feed_cfg.repos_query_mode.query_mode "GitLabGroup" \
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "mygitlabsrc"

Create a Jira PAT to Authenticate Faros

You can create a Personal Access Token in Jira to enable read-only access from the Faros connector

  1. In Jira, select your profile picture at the top right of the screen, then choose Profile.
  2. Once you access your profile, select Personal Access Tokens in the left-hand menu.
  3. Select Create token.
    • Give your new token a name.
    • The integration user needs application access to Jira, the 'Browse Users' global permission, and the 'Browse Project' and 'View Development Tools' permissions for each project
    • Set the expiry date to "No Expiration"
  4. Click Create.
  5. Run the Faros Jira connector:
./airbyte-local.sh \
  --src farosai/airbyte-faros-feeds-source:latest \
  --src.feed_cfg.feed_name "jira-feed" \
  --src.feed_cfg.domain $JIRA_URL \
  --src.feed_cfg.api_token $JIRA_TOKEN \
  --src.feed_cfg.projects_query_mode.faros_api_url "https://prod.api.faros.ai" \
  --src.feed_cfg.projects_query_mode.faros_api_key $FAROS_API_KEY \
  --src.feed_cfg.projects_query_mode.query_mode "FarosGraph"
  --dst farosai/airbyte-faros-destination:latest \
  --dst.edition_configs.api_key $FAROS_API_KEY \
  --dst.edition_configs.api_url "https://prod.api.faros.ai" \
  --dst.edition_configs.edition "cloud" \
  --connection-name "myjirasrc"

Updated about 2 months ago