GitHub Action using ImageMagick: Create a PNG and commit it to the repo - github

Background
I'm maintaining a forked repository that contains a LaTeX project. The README.md contains a .png preview of the .pdf compiled from the example .tex file included in the repo. I often forget to update the .png version using the following ImageMagick command:
magick convert -density 1200 -background white -alpha off Twenty-Seconds-Icons_cv.pdf -quality 90 Twenty-Seconds-Icons_cv.png
Thus I would like to automate this process using GitHub Actions.
Workflow
I think the main.yml file should look something like this, however, I don't fully understand what I am doing.
name: PDF to PNG
on:
push:
branches:
- kaspar
pull_request:
branches:
- kaspar
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Install ImageMagick
run: sudo apt install imagemagick # Seems to work and already be included with ubuntu-latest...
- name: Convert PDF to PNG
run: # How?
- name: Commit the updated PNG
run: # How?
Output:
Run sudo apt install imagemagick
sudo apt install imagemagick
shell: /usr/bin/bash -e {0}
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
imagemagick is already the newest version (8:6.9.10.23+dfsg-2.1ubuntu11.2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Questions
It seems like I can just run bash shell commands with run: ; is this true?
How/where can I access the .pdf file in the repo from inside the GitHub Actions Ubuntu machine?
Where should I save the .png file, is there a ~ directory?
How can I commit the .png file generated by ImageMagick to the repo?
Would git commit <file> -m "updated png version of the CV" work?

I figured out a solution to my problem:
I forgot to checkout the repository
ghostscript needed to be installed explicitly
The security policy for ImageMagic needs to be edited for it to process PDFs
On Ubuntu the ImageMagick command doesn’t work for 1200 dpi (900 is ok in my case), on Windows, this works just fine
I found out how to commit and push files
name: PDF to PNG
on:
push:
branches:
- kaspar
pull_request:
branches:
- kaspar
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Install ghostscript
run: sudo apt install ghostscript
- name: Change ImageMagick security policy
run: |
DQT='"'
SRC="rights=${DQT}none${DQT} pattern=${DQT}PDF${DQT}"
RPL="rights=${DQT}read\|write${DQT} pattern=${DQT}PDF${DQT}"
sudo sed -i "s/$SRC/$RPL/" /etc/ImageMagick-6/policy.xml
- name: Convert PDF to PNG
run: convert -density 900 -background white -alpha off Twenty-Seconds-Icons_cv.pdf -quality 90 Twenty-Seconds-Icons_cv.png
- name: Commit PNG
id: commit
run: |
git config --local user.email "action[bot]#github.com"
git config --local user.name "github-actions[bot]"
git add Twenty-Seconds-Icons_cv.png
if [-z "$(git status --porcelain)"]; then
echo "::set-output name=push::false"
else
git commit -m "[bot] updated Twenty-Seconds-Icons_cv.png"
echo "::set-output name=push::true"
fi
shell: bash
- name: Push Commit
if: steps.commit.outputs.push == 'true'
uses: ad-m/github-push-action#master
with:
github_token: ${{ secrets.SECRET_TOKEN }}

Related

Jmeter upload test artifacts on GIT

Hello I want to upload the HTML file generated from the execution of my Jmeter, unfortunately I'm encountering an error upon executing my script. Your response is highly appreciated. Thank you
Here's my YAML file.
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
inputs:
choice:
type: choice
description: Environment
options:
- test
- dev
- uat
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: setup-jmeter
run: |
sudo apt-get update
sudo apt install curl -y
sudo apt install -y default-jdk
sudo curl -O https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.3.tgz
sudo tar -xvf apache-jmeter-5.3.tgz
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib && sudo curl -O https://repo1.maven.org/maven2/kg/apc/cmdrunner/2.2.1/cmdrunner-2.2.1.jar
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib/ext && sudo curl -O https://repo1.maven.org/maven2/kg/apc/jmeter-plugins-manager/1.6/jmeter-plugins-manager-1.6.jar
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib && sudo java -jar cmdrunner-2.2.1.jar --tool org.jmeterplugins.repository.PluginManagerCMD install-all-except jpgc-hadoop,jpgc-oauth,ulp-jmeter-autocorrelator-plugin,ulp-jmeter-videostreaming-plugin,ulp-jmeter-gwt-plugin,tilln-iso8583
- name: run-jmeter-test
run: |
echo "choice is ${{ github.event.inputs.choice }}" / ${{ inputs.choice }}
$GITHUB_WORKSPACE/apache-jmeter-5.3/bin/./jmeter.sh -n -t testGIT.jmx -Jchoice="${{ github.event.inputs.choice }}" -l result.jtl -e -o $GITHUB_WORKSPACE/html/test
- name: Upload Results
uses: actions/upload-artifact#v2
with:
name: jmeter-results
path: result.jtl
- name: Upload HTML
uses: actions/upload-artifact#v2
with:
name: jmeter-results-HTML
path: index.html
Expected Result:
I should able to see 2 entries for the result one for jmeter-results and the other one is jmeter-results-HTML.
Screenshot:
Note: the index.html generated from my local this is what I want to display from my execution
You're creating HTML Reporting Dashboard under html/test folder and trying to upload index.html file from the current folder. I believe you need to change the artifact path to
path: html/test/index.html
It doesn't make sense to archive index.html alone, it relies on the content and sbadmin2-1.0.7 folders so it's better to consider uploading the whole folder otherwise the dashboard will not be usable.
According to JMeter Best Practices you should always be using the latest version of JMeter so consider upgrading to JMeter 5.5 (or whatever is the latest stable version available at JMeter Downloads page)

fatal: Commit found, use 'git ftp push' to sync

I am getting this error in my circleci yaml file when uploading to my ftp site via my github, there is definitely a change for it to commit but i'm getting this error:
HEAD is now at 76a63d8 Updated config.yml
fatal: Commit found, use 'git ftp push' to sync. Exiting.
..
Even though I have that in my yaml file:
version: 2
jobs:
deploy:
docker:
- image: circleci/node:8-browsers
working_directory: ~/repo
steps:
- checkout
- run: npm install
- run:
name: Deploy Master Branch
command: |
sudo apt-get update
sudo apt-get -qq install git-ftp
echo "Deploying project ..."
git ftp init --user ${username} --passwd ${password} ${ftp_location}
- git reset --hard
- git ftp push
workflows:
version: 2
master-deploy:
jobs:
- deploy:
filters:
branches:
only: main
yes i am also getting the same error and i fixed it with this code
sudo apt-get update
sudo apt-get -qq install git-ftp
git config git-ftp.user ${username}
git config git-ftp.url ${ftp_location}
git config git-ftp.password ${password}
git ftp push
Please follow this code

I don't have access my codes in the runner in GitHub Actions

I created the following "main.yml" file.
name: Deploy
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: test
run: ls -al && cd .. && ls -al
- name: Create SSH key
run: |
mkdir -p ~/.ssh/
echo "$SSH_PRIVATE_KEY" > ~/.ssh/private.key
sudo chmod 600 ~/.ssh/private.key
ssh-keyscan -H ${{secrets.SSH_HOST}} > ~/.ssh/known_hosts
echo "Host ${{secrets.SSH_HOST}}
User ${{secrets.SSH_USER}}
IdentityFile ~/.ssh/private.key" > ~/.ssh/config
cat ~/.ssh/config
shell: bash
env:
SSH_PRIVATE_KEY: ${{secrets.SSH_PRIVATE_KEY}}
- name: test-remote
run: rsync -r ${{secrets.SSH_USER}}#${{secrets.SSH_HOST}}:~/${{secrets.SSH_HOST}}
- name: Deploy with rsync
run: cd .. && ls -al && rsync -avz ./ ${{ secrets.SSH_USER }}#${{ secrets.SSH_HOST }}:/var/www/${{ secrets.SSH_HOST }}
However, I cannot access my codes in the github repository as seen in the following output in the runner.
Maybe I'm using the rsync command incorrectly, so I tried to output with ls and even to output from its parent directory. How do you think I can solve it?
Junior things... I forgot to checkout in the beginning. I added checkout to the beginning of the steps as below and the problem was solved.
- name: Checkout
uses: actions/checkout#main

How to Set Gihub Secret with a Binary File for Github Workflow?

I would like to add one of my API configuration file (binary.file) to the Github secret (MY_BINARY_SECRET). Then it will be read and wrote to binary.file again in the workflow:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Install System
run: |
sudo apt-get update
sudo apt-get install -y pip python3.8-venv libcurl4-openssl-dev
- name: Set up configurations
shell: bash
run: |
echo "${{ secrets.MY_BINARY_SECRET }}" > binary.file
python3 .... # the python script will need binary.file to complete authentication
However, I tried many hours with different ways to copy the binary content to the Github Secret, but all failed. I tried pbcopy, less, cat. Does anyone know how to write a binary file via Github Secret in the github actions? Or a better solution?
Thank you!
(Extending my comment):
Use base64 to encode binary string to text and decode it back to binary. This is pretty standard trick.
First, encode at home:
echo "$MY_BINARY_SECRET" | base64 --wrap=0 > secret.b64
--wrap=0 to make the output text one long line; useful for echo below.
Upload text file secret.b64 as the secret to GitHub. Decode it using
echo -n "${{ secrets.MY_BINARY_SECRET }}" | base64 --decode > binary.file
Advice: first try decoding locally and compare with the original string. Must be the same.

How to self-host Read the Docs using GitHub Pages

How can I setup a CI/CD workflow with gitlab (or GitHub Actions) that generates my own Read the Docs site and is hosted for free using gitlab pages?
Is there a fork-ready example repo on gitlab or github that I can use to self-generate and self-host my own Read the Docs site?
You can host a sphinx-powered site (optionally using the Read the Docs theme) on GitHub Pages using GitHub Actions to wrap sphinx-build and push your html static assets to your GitHub Pages source, such as the gh-pages branch..
You need to define a GitHub Actions workflow to execute a build script.
Here's an example workflow that will execute buildDocs.sh every time there's a push to master
name: docs_pages_workflow
# execute this workflow automatically when a we push to master
on:
push:
branches: [ master ]
jobs:
build_docs_job:
runs-on: ubuntu-latest
container: debian:buster-slim
steps:
- name: Prereqs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
apt-get update
apt-get install -y git
git clone --depth 1 "https://token:${GITHUB_TOKEN}#github.com/${GITHUB_REPOSITORY}.git" .
shell: bash
- name: Execute script to build our documentation and update pages
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: "docs/buildDocs.sh"
shell: bash
And here's an example buildDocs.sh script that's executed by the workflow above:
#!/bin/bash
################################################################################
# File: buildDocs.sh
# Purpose: Script that builds our documentation using sphinx and updates GitHub
# Pages. This script is executed by:
# .github/workflows/docs_pages_workflow.yml
#
# Authors: Michael Altfield <michael#michaelaltfield.net>
# Created: 2020-07-17
# Updated: 2020-07-17
# Version: 0.1
################################################################################
###################
# INSTALL DEPENDS #
###################
apt-get update
apt-get -y install git rsync python3-sphinx python3-sphinx-rtd-theme
#####################
# DECLARE VARIABLES #
#####################
pwd
ls -lah
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
##############
# BUILD DOCS #
##############
# build our documentation with sphinx (see docs/conf.py)
# * https://www.sphinx-doc.org/en/master/usage/quickstart.html#running-the-build
make -C docs clean
make -C docs html
#######################
# Update GitHub Pages #
#######################
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}#users.noreply.github.com"
docroot=`mktemp -d`
rsync -av "docs/_build/html/" "${docroot}/"
pushd "${docroot}"
# don't bother maintaining history; just generate fresh
git init
git remote add deploy "https://token:${GITHUB_TOKEN}#github.com/${GITHUB_REPOSITORY}.git"
git checkout -b gh-pages
# add .nojekyll to the root so that github won't 404 on content added to dirs
# that start with an underscore (_), such as our "_content" dir..
touch .nojekyll
# Add README
cat > README.md <<EOF
# GitHub Pages Cache
Nothing to see here. The contents of this branch are essentially a cache that's not intended to be viewed on github.com.
If you're looking to update our documentation, check the relevant development branch's 'docs/' dir.
For more information on how this documentation is built using Sphinx, Read the Docs, and GitHub Actions/Pages, see:
* https://tech.michaelaltfield.net/2020/07/18/sphinx-rtd-github-pages-1
EOF
# copy the resulting html pages built from sphinx above to our new git repo
git add .
# commit all the new files
msg="Updating Docs for commit ${GITHUB_SHA} made on `date -d"#${SOURCE_DATE_EPOCH}" --iso-8601=seconds` from ${GITHUB_REF} by ${GITHUB_ACTOR}"
git commit -am "${msg}"
# overwrite the contents of the gh-pages branch on our github.com repo
git push deploy gh-pages --force
popd # return to main repo sandbox root
I wrote an article that describes how to run your own Read the Docs site on GitHub Pages that describes the above files in more detail.
I adapted #Michael Altfield's solution into a single GitHub Action:
name: docs_pages_workflow
on:
push:
branches: [ main ]
jobs:
build_docs_job:
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout
uses: actions/checkout#v2.3.4
- name: Set up Python
uses: actions/setup-python#v2.2.1
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install -U sphinx
python -m pip install sphinx-rtd-theme
- name: make the sphinx docs
run: |
make -C docs clean
make -C docs html
- name: Init new repo in dist folder and commit
run: |
cd docs/build/html/
git init
touch .nojekyll
git add -A
git config --local user.email "action#github.com"
git config --local user.name "GitHub Action"
git commit -m 'deploy'
- name: Force push to destination branch
uses: ad-m/github-push-action#v0.5.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: gh-pages
force: true
directory: ./docs/build/html
Note, my Makefile builds to build not _build directory. The last line with directory: is saying to push from the .docs/build/html directory where we just created the new Git repo. This avoids his rsync and pushd commands. Otherwise the logic follows #Michael Altfield's solution.