github actions publish gem package suddenly started to fail - github

I used to publish gem packages to GitHub Packages using the following GitHub Actions and it was always successful.
name: Deploy to Github Packages
on:
release:
types:
- published
env:
ORGANIZATION: MYGITHUBNAME
RELEASE_TAG_NAME: ${{ github.event.release.tag_name }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Set up JDK 8
uses: actions/setup-java#v3
with:
java-version: 8
distribution: temurin
- name: gradlew build
run: |
VERSION=$(echo $RELEASE_TAG_NAME | sed -E 's/(v)(.*)/\2/')
./gradlew gem -Pversion=$VERSION
- name: Set up Ruby
uses: actions/setup-ruby#v1
with:
ruby-version: 3.0
- name: Setup Release Credentials
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
run: |
mkdir -p $HOME/.gem
touch $HOME/.gem/credentials
chmod 600 $HOME/.gem/credentials
echo "---" >$HOME/.gem/credentials
echo ":github: Bearer ${GITHUB_TOKEN}" >> $HOME/.gem/credentials
- name: Publish Gem to GitHub Packages
run: |
PACKAGE=$(find build/gems -type f | sort | tail -n 1)
gem push --KEY github --host https://rubygems.pkg.github.com/${ORGANIZATION} ${PACKAGE}`
However, with the repository I created today, it suddenly stopped working.
Also, when I create it in an existing repository, it succeeds.
The error message when it fails is:
Pushing gem to https://rubygems.pkg.github.com/MYGITHUBNAME...
Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured.
Error: Process completed with exit code 1.
When I use PAT to push the gem from my local environment, it succeeds, but it doesn't appear in the "packages" of the repository.
If anyone knows what is causing this, please let me know.
Thank you.
Unified repository and gem names (failed)
I cloned the repository where gem push was successful and tried with a different repository and Gem name (failed)

This was solved!
Apparently, an item called Workflow permissions has been added to the repository's Settings > Actions > General, and it seems that the existing repository has Read and Write permissions, but the new repository has read-only permissions, hence the permission denied error.
After changing this to Read and Write, I was able to push packages.
If this information is incorrect, could someone please correct it?
Thank you.

Related

Permission denied error in github actions

I have written a github action to retrieve the changed sql files and lint those changed files using sqlfluff.
Here is my github action code:
name: files_lint
on:
- pull_request
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout#v2
- name: Install Python
uses: "actions/setup-python#v2"
with:
python-version: "3.7"
- name: install sqlfluff
run: "pip install sqlfluff"
- name: Get changed .sql files
id: linting
run: some code to get the changed files
- name: Linting files started
id: sql_linting
if: steps.linting.outputs.lintees != ''
shell: bash -l {0}
run: ${{ steps.linting.outputs.lintees }} > sqlfluff fix --force
But when I run ${{ steps.linting.outputs.lintees }} > sqlfluff fix --force on the changed sql files in the above github action, I'm getting an error
/home/runner/work/_temp/a41i1c89a4.sh: line 1: test.sql: Permission denied
Error: Process completed with exit code 126.
You can’t redirect files like this:
run: ${{ steps.linting.outputs.lintees }} > sqlfluff fix --force
This is attempting to write the output of whatever that command is - but I’d guess it’s a list of files rather than a command?
You should pass as parameters (assuming it’s a list of files):
run: sqlfluff fix --force ${{ steps.linting.outputs.lintees }}
Also I presume you’re going to do something with it afterwards? If not the fixed files will not do anything. If you just want to check the files sqlfluff lint would be better than sqlfluff fix (and catches more issues as sqlfluff fix only looks at rules it can fix).
For all developers who created shell script (.sh) locally on Windows or in Windows Subsystem Linux (WSL), or cloned the git repository without knowing on which file system this shell script was created, make sure that shell script is Linux executable!
Linux
chmod +x script.sh
Windows
git update-index --chmod=+x script.sh
Finally, don't forget to push your changes.
git add script.sh
git commit -m'Making script.sh executable'
git push

GitHub Action checkout from specific directory

I am trying to upload a repo to server via ftp on push to master branch. I have it set up and working. However in the repo there is a folder /public. I only want to upload the files in this folder to the server. Not other files or the folder itself. I have tried to set up a working directory for the job but this doesn't seem to do the trick.. any ideas?
on:
push:
branches:
- master
name: 🚀 Deploy website on push
jobs:
ftp-web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./public
steps:
- name: 🚚 Get latest code
uses: actions/checkout#v2.4.0
working-directory: ./public
with:
fetch-depth: 2
- name: 📂 Sync files
uses: SamKirkland/FTP-Deploy-Action#4.2.0
with:
server: ****
username: ****
password: ${{ secrets.prod_ftp_password }}
server-dir: public_html/
Checking out only one directory is not possible, but has been requested in the actions/checkout repository before: https://github.com/actions/checkout/issues/483
There's an action to check out specific files, but I haven't tried it and I'm not sure if it does what you want: https://github.com/marketplace/actions/checkout-files
You might want to ask yourself why you're trying to limit the number of files transferred. Is it because you're concerned about traffic? Or because of the input expected in the subsequent action?
If it's the latter, you could also manually "fix" the structure by running some mv and rm commands.

Updating GitHub issues from GitHub Actions

I was trying to make a GitHub action using some simple scripts (which I already use locally) that I would like to run inside a docker container.
A new issue should trigger the event to update the said issue with its content based on some processing. An example of this might be:
Say I have a list of labels defined in my script and it checks the issue's title and adds a label to the issue.
I'm still reading the GitHub Action's documentation so I may be not completely informed but the issue I seem to have is that in my local machine these scripts use gh cli for doing such tasks (eg. adding labels). So I was wondering if I need to have the gh installed in that docker container or is there a better way to update the issue? I'm very much willing to make these scripts from scratch again using the GitHub's event payloads and stuff as long as I don't have to write in TypeScript.
I've looked around the documentation and couldn't find anything that talked about updating issues. Also couldn't find a similar question being asked here; it may be that I've missed something so if that is the case direct me to relevant material and I would very much appreciate it.
An option could be (as you said) to install GH in that docker container, and then run GH commands.
Example using a container:
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker://myrepoandimagewithghinstalled
steps:
- name: Github CLI Authentication
run: gh auth login --hostname <your hostname>
- name: Github CLI commands execution samples
run: |
gh command1
gh command2
gh command3
Another option could be to install GH directly on the OS (for exemple ubuntu-latest), authenticate, and then use the "run" option to execute GH command.
Example installing GH on the OS:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Install Github CLI
run: |
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key C99B11DEB97541F0
sudo apt-add-repository https://cli.github.com/packages
sudo apt update
sudo apt install gh
- name: Github CLI Authentication
run: gh auth login --hostname <your hostname>
- name: Github CLI commands execution samples
run: |
gh command1
gh command2
gh command3
Finally, you could also create a script consuming the Github API service to update an ISSUE and execute the script using the run option.
Example to execute a Python script in your workflow:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout repo content
uses: actions/checkout#v2 # checkout the repository content to github runner.
- name: setup python
uses: actions/setup-python#v2
with:
python-version: 3.8 #install the python needed
- name: execute py script # run the run.py to get the latest data
run: |
python run.py
env:
key: ${{ secrets.key }} # if run.py requires passwords..etc, set it as secrets
- name: export index
.... # use crosponding script or actions to help export.

Github actions only on first push

I'm working on a github action that creates an app on ArgoCD. The problem is that I want to execute it only once, the first time that it gets push with the k8s yamls.
Is there any way to restrict the github action to the first push on the repo?
I have been looking to the github triggers, but I was not able to find any relation.
This is a sample of the action:
on: push
name: deploy-argo-app
jobs:
deploy-argo-app:
name: Deploy new app on ArgoCD
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Install Argo Cli
run: |
VERSION=$(curl --silent "https://api.github.com/repos/argoproj/argo-cd/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/')
sudo curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/$VERSION/argocd-linux-amd64
sudo chmod +x /usr/local/bin/argocd
- name: Create app
run: |
argocd app create guestbook --repo https://github.com/argoproj/argocd-example-apps.git --path guestbook --dest-server https://kubernetes.default.svc --dest-namespace default
I found kind of a workaround.
on: push
name: deploy
jobs:
deploy:
if: github.run_number == 1
Basically "github.run_number" gives you the push number. It will work only on the first push and then it will be ignore.
I have been looking for something similar, but I needed to know the first push on any given branch.
First, you can see everything available to you on your Github context by running this worfklow:
- name: Dump GitHub context
id: github_context_step
run: echo '${{ toJSON(github) }}'
I was able to see that on the first push to any branch the
github.event.before = 0000000000000000000000000000000000000000
and
github.event.created = true
whereas for any subsequent push to that same branch the github.event.before will give a numerical reference to the previous commit, and the github.event.created will be false.
Slightly related, but maybe still useful for people who find themselves on the same hunt I was on!

Github actions, 401 unauthorized when installing a Github Package with npm or yarn

When I try to install my npm modules from a GitHub action I get the following error:
npm ERR! 401 Unauthorized - GET https://npm.pkg.github.com/#xxxx%2fxxxx-analytics - Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured.
Before you comment, I have configured the .npmrc correctly with the scope and access token, and everything works fine when installing the private package locally.
Here is my GitHub workflow action:
name: JavaScript workflow
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Use Node.js 12.x
uses: actions/setup-node#v1
with:
node-version: '12.x'
- name: npmrc
run: cat .npmrc
- name: npm install
run: |
npm install
env:
CI: true
NPM_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
here is my .npmrc
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=XXXXXXXXX
#colonynetworks:registry=https://npm.pkg.github.com
//npm.pkg.github.com:_authToken=XXXXXXXXX
always-auth=true
#react-admin:registry=https://registry.marmelab.com
//registry.marmelab.com:
_auth=XXXXXXXXX
email=software#XXXXXXXXX.com
always-auth=true
It's a private repo and the authTokens are currently hardcoded in the .npmrc file.
However while trying to find a solution for this, I did come across this random comment from a Github staff member: https://github.community/t/netlify-getting-401-from-github-package-registry-with-auth-token/16415/3
It's a bit vague, but it sounds like it doesn't accept a hardcoded authToken in the .npmrc file.
So first thing I tried was to use our env variable instead like so:
#xxxx=https://npm.pkg.github.com
//npm.pkg.github.com:_authToken=${NPM_AUTH_TOKEN}
The env variable is correct in our Github repo secrets, and supplied by the workflow.
However this still resulted in the same 401 Unauthorized error.
From looking at other solutions I then tried to generate the .npmrc manually inside the Github action before the install step, like so:
- name: npmrcgen
run: |
echo "//npm.pkg.github.com/:_authToken=XXXXXXX" > .npmrc
echo "#xxxxx=https://npm.pkg.github.com/" >> .npmrc
echo "#react-admin:registry=https://registry.marmelab.com" >> .npmrc
echo "//registry.marmelab.com:" >> .npmrc
echo "_auth=XXXXXXX" >> .npmrc
echo "email=software#xxxxx.com" >> .npmrc
echo "always-auth=true" >> .npmrc
During the logging step I added, it the _authToken (only for Github) still shows up as ***, and I still got a 401 Unauthorized error.
At this point I wanted to confirm the .npmrc was even being used, so I removed the second private registry we used for marmelab.com, and sure enough, I got an error saying it was no longer able to install their ra-realtime package. This proves the .npmrc file is indeed being read and used by my Github action, but it's not accepting my Github personal access token.
I have tried to generate a new token as well. It has full access to everything under repo: as well as write:packages and read:packages which is what should be required.
Still 401 Unauthorized in the Github action, and still works fine locally.
Lastly I have tried to install it with yarn instead of npm. Unsurprisingly this did not fix it either.
I have seen and tried the following solutions without any success:
Download private module from Github Package Registry via Yarn within a Github Action? Publishing works, but installing is met with '401 Unauthorized'
https://github.com/FerLuisxd/create-npmrc
https://blog.bitsrc.io/install-npm-private-packages-in-ci-cd-with-github-actions-746db95017cc
One thing I have not tried, as I have seen no recommendations on how or this being a good idea, but I have not done an npm login within the Github action. Since no one else has done this, and somehow have it working, I assume this is not necessary.
I contacted GitHub support and they managed to figure out what the problem was.
Github workflows are more strict than local environments and require an extra / before the auth token:
spot the difference:
//npm.pkg.github.com:_authToken=XXXXXXXXX. # broken
//npm.pkg.github.com/:_authToken=XXXXXXXXX # works
adding the extra / before :_authToken= solved the issue for me.
Have a .npmrc file in root of your project.
Content of .npmrc:
registry=https://registry.npmjs.org/
#{scope}:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=********** (Token generated from github)
#{scope} is your organization-name or your username. It is case-sensitive.
Addendum for anyone else who, like me, runs across this question outside the context of GitHub Actions: note that the GitHub package registry uses HTTP Basic Authentication. So if you're trying to test a personal access token and don't want to mess with your .npmrc / .yarnrc, you can pass the credentials in the registry URL, e.g. with yarn:
yarn info "#<github-org>/<repo-name>" \
--registry="https://<github-user>:<token>#npm.pkg.github.com/"
Or with curl:
curl -vL 'http://<github-user>:<token>#npm.pkg.github.com/#<github-org>%2f<repo-name>'
Just use actions/setup-node action.
- uses: actions/setup-node#v3
with:
node-version: 16
cache: "yarn"
registry-url: "https://npm.pkg.github.com"
- name: Build
env:
# also other environment variable
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
yarn
yarn build
This problem is happening due to Visibility access, to be able to delete packages you need Admin Permission.
Even when trying to do this by separate method without github Actions you still need Admin Permission for deletion.
even when you want to use other methods to Delete packages such as following code below, You still need Admin Permissions and PAT token to do so with delete:packages permissions.
I’ve written the kind of cleanup script I was thinking of here using the new packages delete/restore API . 🙂
I had a similar thought. 😃
Here’s a GitHub Action script that can be used to delete untagged images for a specified container package:
- uses: actions/github-script#v3
with:
github-token: ${{ secrets.DELETE_PACKAGES_TOKEN }}
script: |
const response = await github.request("GET /${{ env.OWNER }}/packages/container/${{ env.PACKAGE_NAME }}/versions",
{ per_page: ${{ env.PER_PAGE }}
});
for(version of response.data) {
if (version.metadata.container.tags.length == 0) {
console.log("delete " + version.id)
const deleteResponse = await github.request("DELETE /${{ env.OWNER }}/packages/container/${{ env.PACKAGE_NAME }}/versions/" + version.id, { });
console.log("status " + deleteResponse.status)
}
}
env:
OWNER: user # or orgs/<org name>
PACKAGE_NAME: <package name>
PER_PAGE: 100
OWNER should be either a user name or orgs/ORG_NAME.
DELETE_PACKAGES_TOKEN is a PAT with the delete:packages and write:packages scopes.