I am trying to set up dbt on Google Cloud using this tutorial. The idea is to have a Cloud Build trigger on GitHub repo merge request. However, starting from the first deployment, Cloud Run would use a template image (and it obviouly fails to deploy):
Container image URL selection window
When I manually select the built image, all is fine. But on trigger, it still keeps the one I've selected previously, and I have to manually select the new one.
What am I doing wrong? Thanks!
Related
Backstory:
We have a web app that creates batch jobs in Azure using docker images. In the application configuration there is a parameter to defines which version of the docker image the batch job should use. In our current setup we need to manually change the parameter if we deploy a new version of the docker image.
What I want to do is choose which docker image to use when I create a release for the web app. I already have a working release pipeline where I manually type in which version of the docker image I want to use, but I would like to be able to choose from the available docker images in the repository. The docker images are built in Azure devops and we have a tag on each build with the version number.
Is it possible to achieve this?
I'm working on a Terraform project that will set up all the GCP resources needed for a large project spanning multiple GitHub repos. My goal is to be able to recreate the cloud infrastructure from scratch completely with Terraform.
The issue I'm running into is in order to setup build triggers with Terraform within GCP, the GitHub repo that is setting off the trigger first needs to be connected. Currently, I've only been able to do that manually via the Google Cloud Build dashboard. I'm not sure if this is possible via Terraform or with a script but I'm looking for any solution I can automate this with. Once the projects are connected updating everything with Terraform is working fine.
TLDR; How can I programmatically connect a GitHub project with a GCP project instead of using the dashboard?
Currently there is no way to programmatically connect a GitHub repo to a Google Cloud Project. This must be done manually via Google Cloud.
My workaround is to manually connect an "admin" project, build containers and save them to that project's artifact registry, and then deploy the containers from the registry in the programmatically generated project.
I have read the following answer here about how to run a specific file.
However, let's say I want to run every single aspect of code in the entire repository here that uses MathJax without downloading it.
How would one figure that out and do that? Is it one JavaScript source code that you script?
If so, how do you figure out the URL that you run?
IF you really don't want to download a repository, you might consider using a GitHub Action.
It does access your code on GitHub side, and can execute whatever you need.
A GitHub Action has an API, and use GitHub runner (on GitHub side, so no download on your part) as opposed to self-hosted runner.
A workflow can be anything you need, like for instance github-action-build, to build your project, in a repository-specific fashion.
As an example, github-action-for-latex compile Latex documents, using a Docker image (xu-cheng/latex-docker).
You would need a similar approach, using a Docker image where you can clone that repository, and execute it (because the Docker image would have everything needed to run your project).
And that would be done entirely on GitHub (Azure-based) side.
Build Trigger Setup
Setup a Build Trigger to a GitHub Repo
Trigger Type: Tag
Tag(regex): .*
Cloud Build Configuration file
Substitution variables:
_DEPLOYMENT_ENV: staging
The config is below:
Things I've Looked at
I've checked out the GitHub Applications and Authorizations. Google Cloud Platform is approved
The Google Cloud GitHub Marketplace Plugin is on for the repo and functions. I get a greencheck mark on pull requests after it builds the containers
Current Results and Expectation
I expect there to be tags matched because the repo has some tags. I push some new tags and nothing has changed.
Note: Google Cloud Build's GitHub Marketplace Plugin is still in Alpha, so its features are not reliable and it is not unusual to run into breaking changes... and there is no active support for it.
The screen you are seeing is from Google Cloud Platform -> Cloud Build -> Triggers. It is different and unrelated to the Google Cloud plugin found inside the GitHub Marketplace (I know, it is confusing).
The triggers you setup currently pull in GitHub repos into Google Cloud Source Repositories before your triggers are executed.
The GitHub plugin, I have been having some issues with it these few days and I think they are introducing some new breaking changes on it soon, but when it worked, it does not require any triggers and purely looks at the cloudbuild.yaml file to do builds automatically. I had to create separate scripts inside the cloudbuild.yaml to setup different build based on tags / branches (Cloud Build Triggers let you do all these inside the UI), but what you get as part of the GitHub plugin is this "GitHub Checks Events" (the green check / red cross) next to the corresponding commits in GitHub and also a very brief details page. The GitHub plugin is currently acting weird on me and I am in the process of switching over to use Cloud Build Triggers until they have sorted it out.
I think they are working on something to bridge the difference between Google Cloud Build Triggers and the Cloud Build GitHub plugin... just a feeling from the current log messages I see inside Cloud Build...
We'd like to have a separate test and prod project on the Google Cloud Platform but we want to reuse the same docker images in both environments. Is it possible for the Kubernetes cluster running on the test project to use images pushed to the prod project? If so, how?
Looking at your question, I believe by account you mean project.
The command for pulling an image from the registry is:
$ gcloud docker pull gcr.io/your-project-id/example-image
This means as long as your account is a member of the project which the image belongs to, you can pull the image from that project to any other projects that your account is a member of.
Yes, it's possible since the container images are on a per-container basis.