Permission denied error in github actions - github

I have written a github action to retrieve the changed sql files and lint those changed files using sqlfluff.
Here is my github action code:
name: files_lint
on:
- pull_request
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout#v2
- name: Install Python
uses: "actions/setup-python#v2"
with:
python-version: "3.7"
- name: install sqlfluff
run: "pip install sqlfluff"
- name: Get changed .sql files
id: linting
run: some code to get the changed files
- name: Linting files started
id: sql_linting
if: steps.linting.outputs.lintees != ''
shell: bash -l {0}
run: ${{ steps.linting.outputs.lintees }} > sqlfluff fix --force
But when I run ${{ steps.linting.outputs.lintees }} > sqlfluff fix --force on the changed sql files in the above github action, I'm getting an error
/home/runner/work/_temp/a41i1c89a4.sh: line 1: test.sql: Permission denied
Error: Process completed with exit code 126.

You can’t redirect files like this:
run: ${{ steps.linting.outputs.lintees }} > sqlfluff fix --force
This is attempting to write the output of whatever that command is - but I’d guess it’s a list of files rather than a command?
You should pass as parameters (assuming it’s a list of files):
run: sqlfluff fix --force ${{ steps.linting.outputs.lintees }}
Also I presume you’re going to do something with it afterwards? If not the fixed files will not do anything. If you just want to check the files sqlfluff lint would be better than sqlfluff fix (and catches more issues as sqlfluff fix only looks at rules it can fix).

For all developers who created shell script (.sh) locally on Windows or in Windows Subsystem Linux (WSL), or cloned the git repository without knowing on which file system this shell script was created, make sure that shell script is Linux executable!
Linux
chmod +x script.sh
Windows
git update-index --chmod=+x script.sh
Finally, don't forget to push your changes.
git add script.sh
git commit -m'Making script.sh executable'
git push

Related

Github Actions: Run .sh without checking out whole project?

Due to bandwidth limits, I'm trying to checkout a subfolder of my project to Github Actions and found this Action: https://github.com/marketplace/actions/checkout-files
New (broken) Script:
name: Create Build Target
run-name: ${{ github.actor }} is creating ${{ github.ref_name }}
on: create
jobs:
Create:
runs-on: ubuntu-latest
steps:
- name: Checkout to access bash script
uses: Bhacaz/checkout-files#v2
with:
files: CICD
- name: Create Buildtarget info on Unity Cloud Build
env:
api_key: ${{ secrets.api_key }}
org_id: ${{ secrets.org_id }}
project_id: ${{ secrets.project_id }}
branch_name: ${{ github.ref_name }}
credential_id: ${{ secrets.credential_id }}
run: CICD/CreateBuildTarget.sh
I get an error in the Github Actions terminal when triggering the above .yaml file to check out a subdirectory instead of the whole project:
/home/runner/work/_temp/2ddc6165-7186-415a-8d87-bc4d746f659f.sh: line 1: CICD/CreateBuildTarget.sh: Permission denied
60
Error: Process completed with exit code 126.
I had this working before, and made sure the files had the correct permissions:
myUserName#myUserNames-MacBook-Pro cicd % ls -l
total 16
-rwxr-xr-x# 1 myUserName staff 2239 Feb 3 11:17 CreateBuildTarget.sh
-rwxr-xr-x# 1 myUserName staff 449 Feb 3 11:18 DeleteBuildTarget.sh
The only thing I changed was the checkout action.
Before (working):
- name: Checkout to access bash script
uses: actions/checkout#v2.6.0
After (not working):
- name: Checkout to access bash script
uses: Bhacaz/checkout-files#v2
with:
files: CICD
Is what I'm trying to do even possible? For now, I changed the script to trigger on pull request open/reopen, instead of create, but I still want to only check out a subdirectory instead of the whole project.
The action you're using isn't setting the execute bits on the .sh files. It's relatively simple to add them manually after restoring the files, but you might want to fork the action and make it do the right thing.
Try using a Sparse Checkout action instead, it would rely on Git to restore the files and has a lot more expected default behaviors built-in:
gogaille/sparse-checkout
While your files are executable locally (on your machine), they might not have been added to your Git repository with +x (the executable bit).
Which means, once the GitHub Action checks out your file (even limiting itself to one subfolder), said sh script files are not executable.
Try locally to do, using git add --chmod=+x:
cd /path/to/repository
cd CICD
git add --chmod=+x *.sh
git commit -m "Add executable bit in CICD folder"
git push
Then check if your GitHub Action has the same issue.

github actions publish gem package suddenly started to fail

I used to publish gem packages to GitHub Packages using the following GitHub Actions and it was always successful.
name: Deploy to Github Packages
on:
release:
types:
- published
env:
ORGANIZATION: MYGITHUBNAME
RELEASE_TAG_NAME: ${{ github.event.release.tag_name }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Set up JDK 8
uses: actions/setup-java#v3
with:
java-version: 8
distribution: temurin
- name: gradlew build
run: |
VERSION=$(echo $RELEASE_TAG_NAME | sed -E 's/(v)(.*)/\2/')
./gradlew gem -Pversion=$VERSION
- name: Set up Ruby
uses: actions/setup-ruby#v1
with:
ruby-version: 3.0
- name: Setup Release Credentials
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
run: |
mkdir -p $HOME/.gem
touch $HOME/.gem/credentials
chmod 600 $HOME/.gem/credentials
echo "---" >$HOME/.gem/credentials
echo ":github: Bearer ${GITHUB_TOKEN}" >> $HOME/.gem/credentials
- name: Publish Gem to GitHub Packages
run: |
PACKAGE=$(find build/gems -type f | sort | tail -n 1)
gem push --KEY github --host https://rubygems.pkg.github.com/${ORGANIZATION} ${PACKAGE}`
However, with the repository I created today, it suddenly stopped working.
Also, when I create it in an existing repository, it succeeds.
The error message when it fails is:
Pushing gem to https://rubygems.pkg.github.com/MYGITHUBNAME...
Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured.
Error: Process completed with exit code 1.
When I use PAT to push the gem from my local environment, it succeeds, but it doesn't appear in the "packages" of the repository.
If anyone knows what is causing this, please let me know.
Thank you.
Unified repository and gem names (failed)
I cloned the repository where gem push was successful and tried with a different repository and Gem name (failed)
This was solved!
Apparently, an item called Workflow permissions has been added to the repository's Settings > Actions > General, and it seems that the existing repository has Read and Write permissions, but the new repository has read-only permissions, hence the permission denied error.
After changing this to Read and Write, I was able to push packages.
If this information is incorrect, could someone please correct it?
Thank you.

Github Action : failed with "lost connection"

We are Trying to build our gcp instance templated using GitHub Actions.
Where we try to build our java archives and transfer it to GCP instance from GitHub Ubuntu machine.
We have set sshkey to access the GCP instance from Ubuntu machines using
ssh-keygen -t rsa -f ~/.ssh/temp -C root -q -N "" && chmod 400 ~/.ssh/temp && chmod 400 ~/.ssh/temp.pub && echo root:cat ~/.ssh/temp.pub > ~/.ssh/temp-formated.pub && chmod 700 /home/runner/.ssh/temp-formated.pub
We get error response when we try to run the following command
scp -o StrictHostKeyChecking=no -i /home/runner/.ssh/temp ./code-web/target/code.war root#:/opt/code.war
The script worked fine till 5th Dec 2022 and started giving error from 6th Dec 2022.
We used to face some failures but the same worked fine when we re-run the build.
build.yml
# This is a basic workflow to help you get started with Actions
name: build-web
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the develop branch
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
inputs:
short_sha:
description: 'Git sha on which build will be created'
required: true
default: ''
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
with:
ref: ${{ github.event.inputs.short_sha }}
# Build using mvn
- name: Set up JDK 8
uses: actions/setup-java#v2
with:
java-version: '8'
distribution: 'adopt'
cache: 'maven'
- name: Build with Maven
run: mvn --batch-mode --update-snapshots verify
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud#v0
with:
service_account_key: ${{ secrets.GCP_SA_KEY }}
export_default_credentials: true
- name: Gcloud Version
run: gcloud --version
- name: Run build script
run: python ./.github/workflows/build.py ${{ github.event.inputs.short_sha }}
Following is the error log.
We have tried multiple builds for other builds in the other builds in the same repository- those failed too
We have confirmed that the secret is still active.
And the build also is successful hence the file "code.war" exists
Any idea of how to figure out the root cause or any one facing similar issue
###Running: ssh-keygen -t rsa -f ~/.ssh/temp -C root -q -N "" && chmod 400 ~/.ssh/temp && chmod 400 ~/.ssh/temp.pub && echo root:`cat ~/.ssh/temp.pub` > ~/.ssh/temp-formated.pub && chmod 700 /home/runner/.ssh/temp-formated.pub
###Exit Code: 0
###RESPONSE:(b'', b'')
####################################
#########Transfer public key to instance############
###Running: cd ~/ && pwd
###Exit Code: 0
###RESPONSE:(b'/home/runner\n', b'')
###Running: gcloud compute instances add-metadata dummy-temp-web --project=projectname --zone=us-east1-b --metadata-from-file ssh-keys=/home/runner/.ssh/temp-formated.pub
###Exit Code: 0
###RESPONSE:(b'', b'Updated [https://www.googleapis.com/compute/v1/projects/projectname/zones/us-east1-b/instances/dummy-temp-web].\n')
####################################
#Give time for key to propogate
#########copy to remote############
###Running: scp -o StrictHostKeyChecking=no -i /home/runner/.ssh/temp ./code-web/target/code.war root#<ip>:/opt/code.war
###Exit Code: 1
###RESPONSE:(b'', b"Warning: Permanently added '<ip>' (ECDSA) to the list of known hosts.\r\nPermission denied, please try again.\r\nPermission denied, please try again.\r\nroot#<ip>: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\nlost connection\n")
Traceback (most recent call last):
File "/home/runner/work/code/code/./.github/workflows/gcloudBuild.py", line 100, in <module>
execute(f'***copyBuldFileToRemoteCMD***', False)
File "/home/runner/work/code/code/./.github/workflows/gcloudBuild.py", line [30](https://github.com/company/code/actions/runs/3628509641/jobs/6119611343#step:7:31), in execute
raise Exception(f'Sorry, bad exit code***process.returncode***')
Exception: Sorry, bad exit code1
I had also similar issue when I was using ubuntu-latest as job runner in yml file.
Instead of ubuntu-latest I used ubuntu-20.04 then issue resolved for me.
you can try this in your yml file
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-20.04
It is working for me.

Is there a way to log error responses from Github Actions?

I am trying to create a bug tracker that allows me to record the error messages of the python script I run. Here is my YAML file at the moment:
name: Bug Tracker
#Controls when the workflow will run
on:
# Triggers the workflow on push request events
push:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab (for testing)
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
build:
# Self Hosted Runner
runs-on: windows-latest
# Steps for tracker to get activated
steps:
# Checks-out your repository under BugTracker so the job can find it
- uses: actions/checkout#v2
- name: setup python
uses: actions/setup-python#v2
with:
python-version: 3.8
# Runs main script to look for
- name: Run File and collect bug
id: response
run: |
echo Running File...
python script.py
echo "${{steps.response.outputs.result}}"
Every time I run the workflow I can't save the error code. By save the error code, I mean for example... if the python script produces "Process completed with exit code 1." then I can save that to a txt file. I've seen cases where I could save if it runs successfully. I've thought about getting the error in the python script but I don't want to have to add the same code to every file if I don't have to. Any thoughts? Greatly appreciate any help or suggestions.
Update: I have been able to successfully use code in python to save to the txt file. However, I'm still looking to do this in Github if anyone has any suggestions.
You could :
redirect the output to a log file while capturing the exit code
set an output with the exit code value like:
echo ::set-output name=status::$status
in another step, commit the log file
in a final step, check that the exit code is success (0) otherwise exit the script with this exit code
Using ubuntu-latest, it would be like this:
name: Bug Tracker
on: [push,workflow_dispatch]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: setup python
uses: actions/setup-python#v2
with:
python-version: 3.8
- name: Run File and collect logs
id: run
run: |
echo Running File...
status=$(python script.py > log.txt 2>&1; echo $?)
cat log.txt
echo ::set-output name=status::$status
- name: Commit log
run: |
git config --global user.name 'GitHub Action'
git config --global user.email 'action#github.com'
git add -A
git checkout master
git diff-index --quiet HEAD || git commit -am "deploy workflow logs"
git push
- name: Check run status
if: steps.run.outputs.status != '0'
run: exit "${{ steps.run.outputs.status }}"
On windows, I think you would need to update this part:
status=$(python script.py > log.txt 2>&1; echo $?)
cat log.txt

github actions – where are the compilation results?

I have defined a little github action workflow, which is supposed to compile a kss-styleguide from scss.
The steps of that workflow basically trigger building the resulting css and the respective kss-styleguide.
When I run the build process locally on my dev machine the built styleguide is written to the styleguide folder located in the root of my project.
However on github, despite everything being marked off green, I don't know, what or where the resulting files are being written to.
How can I deploy the generated styleguide, if I don't know where it is?
Here's my yaml file for this workflow:
name: Node.js CI
on:
push:
branches: [ mk-node-ci ]
pull_request:
branches: [ mk-node-ci ]
jobs:
build:
name: Build Styleguide
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x]
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- uses: borales/actions-yarn#v2.0.0
with:
cmd: install
env:
NODE_ENV: development
- name: "build CSS files"
uses: borales/actions-yarn#v2.0.0
with:
cmd: "build:css"
- name: "build styleguide files"
uses: borales/actions-yarn#v2.0.0
with:
cmd: "build:styleguide"
Updated 2020.10.14 19:25 GMT
GitHub Actions are performed on a separate "clean" Runner Machine.
actions/checkout#v2 is an action that copies your repository to that machine — typically, to perform tests etc.
In order to get produced results (like modified files) from runner machine back to the original repository, we can use:
(1) upload-artifact action.
(2) git push.
For example, here is my script to modify files from the source directory and put them into the output directory (I run it as an action (bash script): - run: wrap.sh). The script wrap.sh:
echo "Copy directory structure from 'in' to 'out':";
find ./in -type d | while read i;
do
if [ ! -d "${i/in/out}" ]; then
mkdir "${i/in/out}"
echo "${i/in/out}";
fi
done
echo "Wrap files:";
find ./in -type f -name "*" | while read i;
do
echo "${i/in/out}";
cat ./tpl/header.html "$i" ./tpl/footer.html >"${i/in/out}"
git add "${i/in/out}"
done
git config user.name "chang-zhao"
git commit . -m "Wrapping"
git push origin main
Here git add "${i/in/out}" is adding to git a new file with that name. git config user.name "..." is required for the commit to work. git commit . -m "Wrapping" is the commit that puts new files into the repository ("Wrapping" is a name I gave to such commits).
This way files produced on a runner server get pushed to the original repository.