I have a GitHub action with a workflow that uses a container to run it, using a private docker registry (myhostname.com - see below).
jobs:
myjob:
name: My Job
runs-on: [ some-tag-on-runners ]
container:
image: myhostname.com/dir/subdir/image:latest
I tried parameterizing myhostname.com using the {{ $secrets.SOME_SECRET }} feature but it doesn't work. I need to have this workflow in several repos and if the hostname changes I would like to be able to change it in only one place. Can't seem to find anything in the docs at GitHub.
Related
I have my domain and my hosting through one.com and I'm tired of moving individual files through filezilla and wanted to automate that process using github actions.
I guess I'll start out by saying I'm completely new to this and this is my first time trying to setup an action. Basically what I want is to just push the code to my github repo and then it gets build and sent to my host. Kinda like how it is with Netlify.
I stumbled upon this https://github.com/SamKirkland/web-deploy which should do the trick. I've seen tutorials using this method on youtube, but I guess they have a different provider than I do making it easier.
This is what information I have to go off of and I hope it will be enough to set it up:
Host: ssh.one-example.com
Username: one-example.com
Password: the one you chose for SSH in your Control Panel
Port: 22
and this is what I put in the yml file:
on:
push:
branches:
- main
name: Publish Website
jobs:
web-deploy:
name: 🚀 Deploy Website Every Commit
runs-on: ubuntu-latest
steps:
- name: 🚚 Get Latest Code
uses: actions/checkout#v3
- name: 📂 Sync Files
uses: SamKirkland/web-deploy#v1
with:
target-server: ${{ secrets.ftp_host }}
remote-user: ${{ secrets.ftp_username }}
private-ssh-key: ${{ secrets.ftp_password }}
destination-path: ~/destinationFolder/
I've tried having the target server both be ssh.one-example.com (obviously using my own here) and I've tried one-example.com#ssh.one-example.com
But I'm ending up with the following error when the action is running:
Error: Error: The process '/usr/bin/rsync' failed with exit code 255
So safe to say I'm a little lost and would like some guidance on how to make it work. Is it what I'm typing that's the issue, is it the host? And if so how do I fix it?
Any help is much appreciated.
Try an SSH action first, to see if you actually can open an SSH session. This is just for testing the connection: try and execute on the remote server a trivial command (ls, or pwd)
Then, regarding your current original action, check the error messages before your "The process '/usr/bin/rsync' failed with exit code 255".
See as an example of previous error messages SamKirkland/web-deploy issue 5 (not yet resolved).
In the settings I've enabled Github Pages:
I have a Github Action which builds and deploy the page to the branch gh-pages.
name: Continuous Deployment
on:
push:
branches:
- master
schedule:
- cron: '0 0 * * *'
jobs:
build-and-deploy:
name: Build and deploy to Github Pages
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout#v3
- name: Use nodejs
uses: actions/setup-node#v3
with:
node-version: '16.x'
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: Activate dependency cache
uses: actions/cache#v3
id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Caching Gatsby
id: gatsby-cache-build
uses: actions/cache#v3
with:
path: |
public
.cache
key: ${{ runner.os }}-gatsby-build-cache-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-gatsby-build-cache-
- name: Build website
run: yarn build:with-prefix
env:
PATH_PREFIX: '/xyz'
SITE_URL: 'https://xyz.github.io/xyz'
CI: true
- name: Deploy to GitHub Pages
uses: JamesIves/github-pages-deploy-action#v4.3.3
with:
branch: gh-pages
folder: public
clean: true
Now there is another Github Action which seem to deploy my page to Github Actions (using Jakyll):
Now I have two questions, which I couldn't answer by searching the internet:
Do I need this other action pages-build-deployment?
If not, how can I disable it?
If yes, for what it's needed? Am I doing the same work twice?
Do I need this other action pages-build-deployment?
This is provided by GitHub automatically.
As discussed in JamesIves/github-pages-deploy-action #1073:
I think there's a lot of scope for confusion for users of this action, when pages build and deployment will also be running on every push and pushing pages assets.
I think it's impossible to really say right now what that looks like until GitHub announces what these actions intend to do in the long run.
As the post suggests, this is a necessary step that occurs after the push gets made, this was already occurring behind the scenes it just wasn't made visible.
So you can ignore it (and there is no obvious way to disable it).
As for doing the same work twice, the same post adds:
What used to happen is, behind the scenes, we used the github-pages[bot] to pull that branch down and deploy the content to the github-pages environment we create for you automatically.
Now this step happens transparently with this new Actions workflow.
The end goal we’re working on is, if you’d prefer to deploy to the pages environment directly (without committing the content to a gh-pages branch), you’ll be able to do that in your own workflow. This will remove the need for the second workflow that we trigger when you commit to the pages branch.
Context: Dec. 2021 "GitHub Pages: using GitHub Actions for builds and deployments for public repositories".
The initial benefit of this change is enabling you to see your build logs and any errors that may occur which has been a long standing issue for Pages users.
However, in the future this will enable us to give you the ability to fully customize your pages build and deployment workflow to use any static site generator you want without having to push the build output to a special branch of the repository.
I have two workflows, the first one runs a build script and generates an artifact.
The first one is triggered when a pull request is created like this:
name: build
on:
pull_request:
types: [opened, edited, ready_for_review, reopened]
The second flow runs when the first is done, by using the workflow_runtrigger like this:
on:
workflow_run:
workflows: ["build"]
types:
- "completed"
The second flow has to be separate and run after the first one. When done it is supposed to post a comment on the PR that triggered the first workflow, but I am unable to find out how.
According to the Github Action Docs this is one of the typical use cases, as per this qoute:
For example, if your pull_request workflow generates build artifacts, you can create
a new workflow that uses workflow_run to analyze the results and add a comment to the
original pull request.
But I can't seem to find out how. I can get the first workflow's id in the 2nd workflow's context.payload.workflow_run.id, but workflow_run should also have about the pull request, but they`re empty.
What am I doing wrong, and where can I find the necessary info to be able to comment on my created pull request?
You're not doing anything wrong, it's just that the Pull Request datas from the first workflow are not present in the Github Context of the second workflow.
To resolve your problem, you could send the Pull Request datas you need from the first workflow to the second workflow.
There are different ways to do it, for example using a dispatch event (instead of a workflow run), or an artifact.
For the artifact, it would look like something as below:
In the FIRST workflow, you get the PR number from the github.event. Then you save that number into a file and upload it as an artifact.
- name: Save the PR number in an artifact
shell: bash
env:
PULL_REQUEST_NUMBER: ${{ github.event.number }}
run: echo $PULL_REQUEST_NUMBER > pull_request_number.txt
- name: Upload the PULL REQUEST number
uses: actions/upload-artifact#v2
with:
name: pull_request_number
path: ./pull_request_number.txt
In the SECOND workflow, you get the artifact and the Pull Request number from the FIRST workflow, using the following GitHub Apps:
- name: Download workflow artifact
uses: dawidd6/action-download-artifact#v2.11.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
workflow: <first_workflow_name>.yml
run_id: ${{ github.event.workflow_run.id }}
- name: Read the pull_request_number.txt file
id: pull_request_number_reader
uses: juliangruber/read-file-action#v1.0.0
with:
path: ./pull_request_number/pull_request_number.txt
- name: Step to add comment on PR
[...]
I found out about a really nice GitHub Actions Feature called Environments. Using the appropriate syntax a Environment could also be created inside a GitHub Action workflow.yml like this:
environment:
name: test_environment
url: https://your-apps-url-here.com
As the docs state thats a valid way to create GitHub Action Environments:
Running a workflow that references an environment that does not exist
will create an environment with the referenced name.
But inside my current GitHub Action workflow is there a way I dynamically set the url based on a deployment step output? I have a dynamic URL resulting from the deployment process to AWS which I can't define up-front.
The job workflow docs tell us that there's also a way of using expressions inside the url field:
environment:
name: test_environment
url: ${{ steps.step_name.outputs.url_output }}
Now imagine a ci.yml workflow file that uses AWS CLI to deploy a static website to S3, where we used a tool like Pulumi to dynamically create a S3 Bucket inside our AWS account. We can read the dynamically created S3 url using the following command pulumi stack output bucketName. The deploy step inside the ci.yml could then look like this:
- name: Deploy Nuxt.js generated static site to S3 Bucket via AWS CLI
id: aws-sync
run: |
aws s3 sync ../dist/ s3://$(pulumi stack output bucketName) --acl public-read
echo "::set-output name=s3_url::http://$(pulumi stack output bucketUrl)"
working-directory: ./deployment
There are 2 crucial points here: First we should use id inside the deployment step to define a step name we could easily access via step_name inside our environment:url. Second we need to define a step output using echo "::set-output name=s3_url::http://$(pulumi stack output bucketUrl)". In this example I create a variable s3_url. You could replace the pulumi stack output bucketUrl with any other command you'd like or tool you use, which responds with your dynamic environment url.
Be also sure to add a http:// or https:// in order to prevent an error message like this:
Environment URL 'microservice-ui-nuxt-js-hosting-bucket-bc75fce.s3-website.eu-central-1.amazonaws.com' is not a valid http(s) URL, so it will not be shown as a link in the workflow graph.
Now the environment definition at the top of our ci.yml can access the s3_url output variable from our deployment step like this:
jobs:
ci:
runs-on: ubuntu-latest
environment:
name: microservice-ui-nuxt-js-deployment
url: ${{ steps.aws-sync.outputs.s3_url }}
steps:
- name: Checkout
...
Using steps.aws-sync we reference the deployment step directly, since we defined it with the id. The appended .outputs.s3_url then directly references the variable containing our S3 url. If you defined everything correctly the GitHub Actions UI will render the environment URL directly below the finished job:
Here's also a fully working workflow embedded inside a example project.
I'm pretty impressed by the power and simplicity of Concourse. Since my pipelines keep growing I decided to move the tasks to separate files. One of the tasks use a custom Docker image from our own private registry. So, in that task file I have:
image_resource:
type: docker-image
source:
repository: docker.mycomp.com:443/app-builder
tag: latest
username: {{dckr-user}}
password: {{dckr-pass}}
When I do a set-pipeline, I pass the --load-from-vars argument to load credentials etc from a seperate file.
Now here's my problem: I notice that the vars in my pipeline files are replaced with the actual correct values, but once the task runs, the afore mentioned {{dckr-user}} and {{dckr-pass}} are not replaced.
How do I achieve this?
In addition to what was provided in this answer
If specifically you are looking to use private images in a task, you can do the following in your pipeline.yml:
resources:
- name: some-private-image
type: docker
params:
repository: ...
username: {{my-username}}
password: {{my-password}}
jobs:
- name: foo
plan:
- get: some-private-image
- task: some-task
image: some-private-image
Because this is your pipeline, you can use --load-vars-from, which will first get your image as a resource and then use it for the subsequent task.
You can also see this article on pre-fetching ruby gems in test containers on Concourse
The only downside to this is you cannot use this technique when running a fly execute.
As of concourse v3.3.0, you can set up Credential Management in order to use variables from one of the supported credential managers which are currently Vault, Credhub, Amazon SSM, and Amazon Secrets Manager. So you don't have to separate your task files partially in the pipeline.yml anymore. The values you set in the Vault will be also accessible from the task.yml files.
And since v3.2.0 {{foo}} is deprecated in favor of ((foo)).
Using the Credential Manager you can parameterize:
source under resources in a pipeline
source under resource_types in a pipeline
webhook_token under resources in a pipeline
image_resource.source under image_resource in a task config
params in a pipeline
params in a task config
For setting up vault with concourse you can refer to:
https://concourse-ci.org/creds.html
You can always define tasks in a pipeline.yml...
For example:
jobs:
- name: dotpersecond
plan:
- task: dotpersecond
config:
image_resource:
type: docker-image
source:
repository: docker.mycomp.com:443/app-builder
tag: latest
username: {{dckr-user}}
password: {{dckr-pass}}
run:
path: sh
args:
- "-c"
- |
for i in `seq 1000`; do echo hi; sleep 2; done