I'm working on a Terraform project that will set up all the GCP resources needed for a large project spanning multiple GitHub repos. My goal is to be able to recreate the cloud infrastructure from scratch completely with Terraform.
The issue I'm running into is in order to setup build triggers with Terraform within GCP, the GitHub repo that is setting off the trigger first needs to be connected. Currently, I've only been able to do that manually via the Google Cloud Build dashboard. I'm not sure if this is possible via Terraform or with a script but I'm looking for any solution I can automate this with. Once the projects are connected updating everything with Terraform is working fine.
TLDR; How can I programmatically connect a GitHub project with a GCP project instead of using the dashboard?
Currently there is no way to programmatically connect a GitHub repo to a Google Cloud Project. This must be done manually via Google Cloud.
My workaround is to manually connect an "admin" project, build containers and save them to that project's artifact registry, and then deploy the containers from the registry in the programmatically generated project.
Related
I have a simple Spring Boot REST app that uses Mongo Atlas as the database and I have a couple of environment variables to pass to the project for the connection URL. I have defined this in the src/main/resources/application.properties which is the standard Spring profile for storing such properties. Here's the property name and value.
spring.data.mongodb.uri=mongodb+srv://${mongodb.username}:${mongodb.password}#....
I use VSCode for local dev and use a launch.json file which is not committed to my github repo to pass these values and I can run this locally. I was able to deploy this app successfully to Heroku and setup these two values in the Heroku console in my App settings and it all works fine on Heroku also. I am trying to deploy the same app to GCP App Engine but I could not find an easy way to pass these values. All the help articles seem to indicate that I need to use some gcp datastore and some cloud provider specific code in my app. Or use some kind of a github action script. That seems a little bit involved and I wanted to know if there is an easy way of passing these values to the app via gcp settings (just like in Heroku) without polluting my repo with cloud provider specific code or yml files.
Greetings from Brazil!
I have an app in github which I am deploying to a cloud service. I want to deploy this same app to other services such as Heroku, AWS and/or IBM Cloud, using Github diff changes (i.e. when I update the repo it automatically updates the cloud app - like magic). Currently GitHub diff changes works fine with streamlit share and heroku, but I have separate repos.
My questions is that: can I deploy an app to multiples services from just one repository in GitHub?
Irrelevant for the question: currently the app is Python3 and I share the app in streamlit share and Heroku, using separate repos. My question, however, is app agnostic.
You can use GitHub Actions to define your deployment workflows.
You can deploy to various cloud providers using available actions/operators:
Amazon ECS
Azure
Heroku
Your project can define a workflow for each cloud provider and, within each workflow, decide when the deployment occurs (automatically on every push, only selected branches or manually - pushing a button).
We run Google Cloud Functions (python), which require to be deployed from Google Cloud Source Repository. Since all the code is stored on GitHub we resort to first mirroring GitHub into Source Repository. Although this only requires a few mouse clicks, it becomes a burden to repeat over 3+ projects (dev, staging, production) times 5+ repos (5+ apps).
I am looking to automate the mirroring config, preferably to add into the Terraform automation we already use, into a hands-off project configuration. Does the Google API support this mirroring automation? So far on my Google Cloud expedition everything was available in their API!
I fail to find Terraform examples though, and would appreciate a tip.
Come to think of it, if I can take Source Repository out of the equation, that would be just fine with me too. After all, I only use it as a pass-through / empty shell.
The Cloud Source Repository API includes a Repo resource that has a Mirror Config object where you could type in your Github's URL, webhook and credentials to automate this procedure. I would initially test it with the create method, but if you have an existing Cloud Source Repository I believe the patch method will also be worth exploring.
Additionally, there is an open Feature Request in order to connect a repository via the Cloud Build GitHub App that I recommend you to star and follow, as it could further ease your automation needs.
I have many projects which uses Google Cloud Build + Github build pipeline setup. However, there is this one project, which I cannot create a webhook in Github for.
It used to work - but commits to the repository doesn't trigger the build process any more. I deleted the trigger and added it again - but the webhook in Github is not created automatically for this project.
When I run the trigger manually, it picks the wrong, but fixed commit which I did before an year.
Any clue?
Could you try delete a repository on Cloud Source Repositories and setup Google Cloud Build again ?
See:
https://cloud.google.com/cloud-build/docs/running-builds/automate-builds
Note: For external repositories, such as GitHub and Bitbucket, you must have owner-level permissions for the Cloud Platform project with which you're working. When you set up a build trigger with an external repository for the first time, you'll need to set up authorization with that repository.
After you've set up your external repository, Cloud Source Repository creates a mirror of your repository.
https://source.cloud.google.com
https://cloud.google.com/source-repositories/docs/deleting-a-repository
https://cloud.google.com/source-repositories/docs/mirroring-a-github-repository
I am experiencing the same issue. I can create a trigger for a repo, but I cannot connect the repo automatically to cloud build. We also have many projects, and this manual labor is sort of annoying.
Is there any (under the hood) github/gcloud api available in which I can connect a github repo to cloud build? I am aware that this can only be done by someone with admin privileges on a repo or organization in github.
After this, I will be able to run the command gcloud build triggers create github [NAME]
We installed the Google Cloud Build GitHub app. We then created some Build Triggers with the Google Cloud Build web user interface. This worked for a while.
Recently we pushed new branches to our GitHub repositories and tried to create Google Cloud Build Triggers for those branches. The Trigger Settings page says "No branch matches" even though we are 100% sure that the branch exists on GitHub.
How can we refresh the branch listings in the Google Cloud Build Trigger Settings page?
We have tried logging in/out of Google Cloud Build and GitHub. We have also tried uninstalling and re-installing the Google Cloud Build GitHub app. We have also tried simply waiting for a few hours.
The problem was that, while configuring Cloud Build with GitHub, we had pushed too many buttons.
This is an overview of what we needed to reset.
In GitHub
Go to the affected repository.
Open its settings.
Delete the Webhook associated with source.developers.google.com/webhook/github.
Delete the Deploy key associated with Google Connected Repository Fingerprint.
In Google Cloud Console (console.cloud.google.com)
Open Source Repositories
Disconnect the mirror of the affected repository.
Open Cloud Build
Delete and recreate the Build Trigger(s) for the affected repository.
Aside re: the Google Cloud Build GitHub App
Setting up specific triggers in Google Cloud Build is orthogonal to using the Google Cloud Build GitHub app. The former does not require the latter. They are different ways to do similar things.
If you had installed the Google Cloud Build GitHub App in your GitHub account because you thought it was required for Google Cloud Build Triggers, then uninstall the Google Cloud Build GitHub App. It works differently than setting up specific Google Cloud Build triggers does, and I found it quite confusing to have both running.