I don't have access my codes in the runner in GitHub Actions - github

I created the following "main.yml" file.
name: Deploy
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: test
run: ls -al && cd .. && ls -al
- name: Create SSH key
run: |
mkdir -p ~/.ssh/
echo "$SSH_PRIVATE_KEY" > ~/.ssh/private.key
sudo chmod 600 ~/.ssh/private.key
ssh-keyscan -H ${{secrets.SSH_HOST}} > ~/.ssh/known_hosts
echo "Host ${{secrets.SSH_HOST}}
User ${{secrets.SSH_USER}}
IdentityFile ~/.ssh/private.key" > ~/.ssh/config
cat ~/.ssh/config
shell: bash
env:
SSH_PRIVATE_KEY: ${{secrets.SSH_PRIVATE_KEY}}
- name: test-remote
run: rsync -r ${{secrets.SSH_USER}}#${{secrets.SSH_HOST}}:~/${{secrets.SSH_HOST}}
- name: Deploy with rsync
run: cd .. && ls -al && rsync -avz ./ ${{ secrets.SSH_USER }}#${{ secrets.SSH_HOST }}:/var/www/${{ secrets.SSH_HOST }}
However, I cannot access my codes in the github repository as seen in the following output in the runner.
Maybe I'm using the rsync command incorrectly, so I tried to output with ls and even to output from its parent directory. How do you think I can solve it?

Junior things... I forgot to checkout in the beginning. I added checkout to the beginning of the steps as below and the problem was solved.
- name: Checkout
uses: actions/checkout#main

Related

npm run prod not actually running in github action despite showing successful

I don't believe that nmp run prod is actually running(?) in my github action despite not throwing any kind of error. The reasons why I believe that are:
If I delete my public/js/app.js file locally and push the change, it doesn't get rebuilt and my production site breaks as there's no app.js file.
If I leave the file in place and push my code to production, it's not minified, and one of the keys I need to reference still contains the dev value.
If I replace the aforementioned key with a different value and run npm run prod locally, then app.js is minified and contains my updated value.
Why would the npm run prod command not work within a github action, and also indicate that it ran successfully?
Here's my entire workflow file:
name: Prod
on:
push:
branches: [ main ]
jobs:
laravel_tests:
runs-on: ubuntu-20.04
env:
DB_CONNECTION: mysql
DB_HOST: localhost
DB_PORT: 3306
DB_DATABASE: testdb
DB_USERNAME: root
DB_PASSWORD: root
steps:
- name: Set up MySQL
run: |
sudo systemctl start mysql
mysql -e 'CREATE DATABASE testdb;' -uroot -proot
mysql -e 'SHOW DATABASES;' -uroot -proot
- uses: actions/checkout#main
- name: Copy .env
run: php -r "file_exists('.env') || copy('.env.example', '.env');"
- name: Install Dependencies
run: composer install -q --no-ansi --no-interaction --no-scripts --no-progress
- name: Generate key
run: php artisan key:generate
- name: Directory Permissions
run: chmod -R 777 storage bootstrap/cache
- name: Clean Install
run: npm ci
- name: Compile assets
run: npm run prod
- name: Execute tests (Unit and Feature tests) via PHPUnit
run: vendor/bin/phpunit
forge_deploy:
runs-on: ubuntu-20.04
needs: laravel_tests
steps:
- name: Make Get Request
uses: satak/webrequest-action#master
with:
url: ${{ secrets.PROD_DEPLOY_URL }}
method: GET
UPDATE:
My suspicion is that running the build process in the action isn't actually updating the repo (actually I'm fairly certain it's probably not as that would likely not be the desired behavior). So then the deploy url that I'm using to push the code is likely just grabbing the repo as-is and deploying it.
I need a way to update only the public folder on the repo with the output of the npm run prod command. Not sure if this is possible, or advisable, but I'm nearly positive that's what's going on.

Jmeter upload test artifacts on GIT

Hello I want to upload the HTML file generated from the execution of my Jmeter, unfortunately I'm encountering an error upon executing my script. Your response is highly appreciated. Thank you
Here's my YAML file.
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
inputs:
choice:
type: choice
description: Environment
options:
- test
- dev
- uat
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: setup-jmeter
run: |
sudo apt-get update
sudo apt install curl -y
sudo apt install -y default-jdk
sudo curl -O https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.3.tgz
sudo tar -xvf apache-jmeter-5.3.tgz
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib && sudo curl -O https://repo1.maven.org/maven2/kg/apc/cmdrunner/2.2.1/cmdrunner-2.2.1.jar
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib/ext && sudo curl -O https://repo1.maven.org/maven2/kg/apc/jmeter-plugins-manager/1.6/jmeter-plugins-manager-1.6.jar
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib && sudo java -jar cmdrunner-2.2.1.jar --tool org.jmeterplugins.repository.PluginManagerCMD install-all-except jpgc-hadoop,jpgc-oauth,ulp-jmeter-autocorrelator-plugin,ulp-jmeter-videostreaming-plugin,ulp-jmeter-gwt-plugin,tilln-iso8583
- name: run-jmeter-test
run: |
echo "choice is ${{ github.event.inputs.choice }}" / ${{ inputs.choice }}
$GITHUB_WORKSPACE/apache-jmeter-5.3/bin/./jmeter.sh -n -t testGIT.jmx -Jchoice="${{ github.event.inputs.choice }}" -l result.jtl -e -o $GITHUB_WORKSPACE/html/test
- name: Upload Results
uses: actions/upload-artifact#v2
with:
name: jmeter-results
path: result.jtl
- name: Upload HTML
uses: actions/upload-artifact#v2
with:
name: jmeter-results-HTML
path: index.html
Expected Result:
I should able to see 2 entries for the result one for jmeter-results and the other one is jmeter-results-HTML.
Screenshot:
Note: the index.html generated from my local this is what I want to display from my execution
You're creating HTML Reporting Dashboard under html/test folder and trying to upload index.html file from the current folder. I believe you need to change the artifact path to
path: html/test/index.html
It doesn't make sense to archive index.html alone, it relies on the content and sbadmin2-1.0.7 folders so it's better to consider uploading the whole folder otherwise the dashboard will not be usable.
According to JMeter Best Practices you should always be using the latest version of JMeter so consider upgrading to JMeter 5.5 (or whatever is the latest stable version available at JMeter Downloads page)

Introducing secret variables to dockerfile on Github Actions

I am trying to configure my etc/pip.conf file to download a private PyPi artifactory while using a secret variable on my dockerfile.
Dockerfile
FROM python
WORKDIR ./app
COPY . /app
RUN pip install --upgrade pip
RUN pip install -r pre-requirements.txt
RUN echo ${{ secrets.PIP }} > etc/pip.conf
RUN pip install -r post-requirements.txt
CMD ["python", "./simpleflask.py"]
docker-image.yml
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Setup JFrog CLI
uses: jfrog/setup-jfrog-cli#v2
env:
JF_ARTIFACTORY_SERVER: ${{ secrets.JFROG_CLI }}
- name: Checkout
uses: actions/checkout#v3
- name: Build
run: |
docker build -t simple-flask .
docker tag simple-flask awakzdev.jfrog.io/docker-local/simple-flask:latest
docker push awakzdev.jfrog.io/docker-local/simple-flask:latest
pretty simple and straightfoward but my pipeline returns the following
Step 6/8 : RUN echo ${{ secrets.PIP }} > etc/pip.conf
---> Running in deb3e3f4167f
/bin/sh: 1: Bad substitution
The command '/bin/sh -c echo ${{ secrets.PIP }} > etc/pip.conf' returned a non-zero code: 2
Error: Process completed with exit code 2.
Edit :
Trying a slightly difference approach and went to install dependencies in the pipeline
my .yml looks like this now
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Setup JFrog CLI
uses: jfrog/setup-jfrog-cli#v2
env:
JF_ARTIFACTORY_SERVER: ${{ secrets.JFROG_CLI }}
- name: Checkout
uses: actions/checkout#v3
- name: install dependencies
run: |
pip config -v list
echo "${{ secrets.PIP }}" > /etc/pip.conf
pip install ganesha-experimental==2.0.1
- name: Build
run: |
docker build -t simple-flask .
docker tag simple-flask awakzdev.jfrog.io/docker-local/simple-flask:latest
docker push awakzdev.jfrog.io/docker-local/simple-flask:latest
but the following error is being returned:
1s
Run pip config -v list
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'global', will try loading '/etc/pip.conf'
For variant 'user', will try loading '/home/runner/.pip/pip.conf'
For variant 'user', will try loading '/home/runner/.config/pip/pip.conf'
For variant 'site', will try loading '/usr/pip.conf'
/home/runner/work/_temp/09382b8f-ce09-4646-816f-fb337f40ad4b.sh: line 2: /etc/pip.conf: Permission denied
Error: Process completed with exit code 1.
I've placed the secret on my .yml file instead.
as for the broken pip permissions I used
sudo chown runner /etc/
echo ${{ secrets.PIP }} > /etc/pip.conf
which resulted in another error with the contents of the pip.conf file (it was configured correctly through secrets)
so I found you can specify the url like so
ganesha_experimental==5.0.0 --find-links=https://awakzdev.jfrog.io/artifactory/

How to self-host Read the Docs using GitHub Pages

How can I setup a CI/CD workflow with gitlab (or GitHub Actions) that generates my own Read the Docs site and is hosted for free using gitlab pages?
Is there a fork-ready example repo on gitlab or github that I can use to self-generate and self-host my own Read the Docs site?
You can host a sphinx-powered site (optionally using the Read the Docs theme) on GitHub Pages using GitHub Actions to wrap sphinx-build and push your html static assets to your GitHub Pages source, such as the gh-pages branch..
You need to define a GitHub Actions workflow to execute a build script.
Here's an example workflow that will execute buildDocs.sh every time there's a push to master
name: docs_pages_workflow
# execute this workflow automatically when a we push to master
on:
push:
branches: [ master ]
jobs:
build_docs_job:
runs-on: ubuntu-latest
container: debian:buster-slim
steps:
- name: Prereqs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
apt-get update
apt-get install -y git
git clone --depth 1 "https://token:${GITHUB_TOKEN}#github.com/${GITHUB_REPOSITORY}.git" .
shell: bash
- name: Execute script to build our documentation and update pages
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: "docs/buildDocs.sh"
shell: bash
And here's an example buildDocs.sh script that's executed by the workflow above:
#!/bin/bash
################################################################################
# File: buildDocs.sh
# Purpose: Script that builds our documentation using sphinx and updates GitHub
# Pages. This script is executed by:
# .github/workflows/docs_pages_workflow.yml
#
# Authors: Michael Altfield <michael#michaelaltfield.net>
# Created: 2020-07-17
# Updated: 2020-07-17
# Version: 0.1
################################################################################
###################
# INSTALL DEPENDS #
###################
apt-get update
apt-get -y install git rsync python3-sphinx python3-sphinx-rtd-theme
#####################
# DECLARE VARIABLES #
#####################
pwd
ls -lah
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
##############
# BUILD DOCS #
##############
# build our documentation with sphinx (see docs/conf.py)
# * https://www.sphinx-doc.org/en/master/usage/quickstart.html#running-the-build
make -C docs clean
make -C docs html
#######################
# Update GitHub Pages #
#######################
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}#users.noreply.github.com"
docroot=`mktemp -d`
rsync -av "docs/_build/html/" "${docroot}/"
pushd "${docroot}"
# don't bother maintaining history; just generate fresh
git init
git remote add deploy "https://token:${GITHUB_TOKEN}#github.com/${GITHUB_REPOSITORY}.git"
git checkout -b gh-pages
# add .nojekyll to the root so that github won't 404 on content added to dirs
# that start with an underscore (_), such as our "_content" dir..
touch .nojekyll
# Add README
cat > README.md <<EOF
# GitHub Pages Cache
Nothing to see here. The contents of this branch are essentially a cache that's not intended to be viewed on github.com.
If you're looking to update our documentation, check the relevant development branch's 'docs/' dir.
For more information on how this documentation is built using Sphinx, Read the Docs, and GitHub Actions/Pages, see:
* https://tech.michaelaltfield.net/2020/07/18/sphinx-rtd-github-pages-1
EOF
# copy the resulting html pages built from sphinx above to our new git repo
git add .
# commit all the new files
msg="Updating Docs for commit ${GITHUB_SHA} made on `date -d"#${SOURCE_DATE_EPOCH}" --iso-8601=seconds` from ${GITHUB_REF} by ${GITHUB_ACTOR}"
git commit -am "${msg}"
# overwrite the contents of the gh-pages branch on our github.com repo
git push deploy gh-pages --force
popd # return to main repo sandbox root
I wrote an article that describes how to run your own Read the Docs site on GitHub Pages that describes the above files in more detail.
I adapted #Michael Altfield's solution into a single GitHub Action:
name: docs_pages_workflow
on:
push:
branches: [ main ]
jobs:
build_docs_job:
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout
uses: actions/checkout#v2.3.4
- name: Set up Python
uses: actions/setup-python#v2.2.1
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install -U sphinx
python -m pip install sphinx-rtd-theme
- name: make the sphinx docs
run: |
make -C docs clean
make -C docs html
- name: Init new repo in dist folder and commit
run: |
cd docs/build/html/
git init
touch .nojekyll
git add -A
git config --local user.email "action#github.com"
git config --local user.name "GitHub Action"
git commit -m 'deploy'
- name: Force push to destination branch
uses: ad-m/github-push-action#v0.5.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: gh-pages
force: true
directory: ./docs/build/html
Note, my Makefile builds to build not _build directory. The last line with directory: is saying to push from the .docs/build/html directory where we just created the new Git repo. This avoids his rsync and pushd commands. Otherwise the logic follows #Michael Altfield's solution.

Github actions scp into VPS via ssh only

This is currently my workflow
name: CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- uses: actions/setup-node#v1
with:
node-version: '10.x'
- run: npm install
- run: npm install -g #angular/cli > /dev/null
- run: ng build --prod
- run: scp -o StrictHostKeyChecking=no -r ./dist/pwa/* user#domain.com://home/user/domain.com/pwa
The above is roughly a translation of what I have on CircleCI. However, obviously the above fails.
CircleCI allowed adding 'SSH Permissions' to a project, so as during setting up build to run, it attaches that to the environment, thus making any ssh commands to the VPS easy.
How can I accomplish a similar approach in Github? Github Actions supports SSH Permissions? If not, is there a workaround?
How do you folks copy files from your workflow builds to an external server via ssh (i.e scp)?
This is what I do, after adding the SSH key to github secrets:
run: |
mkdir -p ~/.ssh
echo "${{ secrets.SSH_KEY }}" > ~/.ssh/id_rsa
chmod 700 ~/.ssh/id_rsa
ssh-keyscan -H domain.com >> ~/.ssh/known_hosts
scp -o StrictHostKeyChecking=no -r ./dist/pwa/* user#domain.com://home/user/domain.com/pwa