Running GitHub scripts directly from Rundeck - github

I have a Windows server running the community version of Rundeck, and I want to be able to run PowerShell scripts directly from my private GitHub repository, by providing the URL of the script.
On GitHub, I have created a personal access token that doesn't expire, because I'm running multiple jobs on Rundeck and I don't want to be modifying each job, every time a token expires.
I've tried various ways to do this but nothing works.
providing the username and password in the URL
providing the username and access token in the URL
https://$username:$token#raw.githubusercontent.com/$username/general/main/rundeck/$script.ps1
https://$username:$password#raw.githubusercontent.com/$username/general/main/rundeck/$script.ps1
I know that at some point GitHub disabled password authentication in the URL, so I'm looking for an alternative way to do it.
If it's possible I don't want to clone the git repository. I want to use a link to the raw script and run it directly.

It works in the following way:
On GitHub Go to your Avatar icon (up to right) > Settings > Developer Settings (left menu) > Personal Access Token > Tokens.
Click on the "Generate new token" button (up to right) > Generate new token (classic). Make sure to create a token with enough rights to read your repository. Save this token value.
On Rundeck, create a new job and add "Script file or URL", on the "File Path/URL" field put the remote path with the following format: https://<your_github_user_name>:<your_user_token>#raw.githubusercontent.com/username/repository/branch/your_script.sh
I made a job definition example that works (tested on Rundeck 4.8.0):
- defaultTab: nodes
description: ''
executionEnabled: true
id: 5f74a444-7b2f-4e50-af96-ca770c9d038e
loglevel: INFO
name: RemoteScriptTest
nodeFilterEditable: false
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- fileExtension: .sh
interpreterArgsQuoted: false
scriptInterpreter: /bin/bash
scripturl: https://user:my_user_token#raw.githubusercontent.com/user/scripts/main/myscript.sh
keepgoing: false
strategy: node-first
uuid: 5f74a444-7b2f-4e50-af96-ca770c9d038e
This example calls a bash script but you can use Powershell to define powershell.exe on the "Invocation String" and .ps1 on the "File extension" text boxes.
Note: if you want to use an option to store the token, use the ${option.myoption} format, here is another example:
- defaultTab: nodes
description: ''
executionEnabled: true
id: 5f74a444-7b2f-4e50-af96-ca770c9d038e
loglevel: INFO
name: RemoteScriptTest
nodeFilterEditable: false
options:
- name: mytoken
value: abc123
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- fileExtension: .sh
interpreterArgsQuoted: false
scriptInterpreter: /bin/bash
scripturl: https://user:${option.mytoken}#raw.githubusercontent.com/user/scripts/main/myscript.sh
keepgoing: false
strategy: node-first
uuid: 5f74a444-7b2f-4e50-af96-ca770c9d038e

Related

GitHub Env Choice

Maybe someone is able to help. I have an example below, where you can choose an env you want to use. But if i have, for example, more then 10 envs, or i want to change them sometimes and i don't want to go to .ci.yml and change it every time. Is there an option not write env names one by one but just to list the env files in a folder ?
name: CI
on:
workflow_dispatch:
inputs:
environment:
type: environment
description: Select the environment
boolean:
type: boolean
description: True or False
choice:
type: choice
description: Make a choice
options:
- foo
- bar
so as you can see, i have foo and bar env. But i don't want to list each env name here.

Azure Yaml pipelines : How to reset a parameter after running the pipeline before "Run new"

I'm using parameters within my yaml pipeline which are set on runtime by the user.
For example:
parameters:
- name: StartAfterDeployment
displayName: 'Start after deployment'
type: boolean
default: true
When the pipeline has ran successfully and (and the user selected false), and the user choose "Run new" on that deployment. The option is selected false instead of the default true.
Is it possible to "reset" the parameters to the default settings when choosing the Run new option or to disable completely the run rew option?
Is it possible to "reset" the parameters to the default settings when choosing the Run new option or to disable completely the run rew option?
I am afraid there is no such way to konw the value of the run rew option.
We have to manually select the value when we rerun the pipeline, we could change the YAML like:
parameters:
- name: StartAfterDeployment
displayName: 'Start after deployment'
type: string
values:
- true
- false
default: true

Github actions, 401 unauthorized when installing a Github Package with npm or yarn

When I try to install my npm modules from a GitHub action I get the following error:
npm ERR! 401 Unauthorized - GET https://npm.pkg.github.com/#xxxx%2fxxxx-analytics - Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured.
Before you comment, I have configured the .npmrc correctly with the scope and access token, and everything works fine when installing the private package locally.
Here is my GitHub workflow action:
name: JavaScript workflow
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Use Node.js 12.x
uses: actions/setup-node#v1
with:
node-version: '12.x'
- name: npmrc
run: cat .npmrc
- name: npm install
run: |
npm install
env:
CI: true
NPM_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
here is my .npmrc
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=XXXXXXXXX
#colonynetworks:registry=https://npm.pkg.github.com
//npm.pkg.github.com:_authToken=XXXXXXXXX
always-auth=true
#react-admin:registry=https://registry.marmelab.com
//registry.marmelab.com:
_auth=XXXXXXXXX
email=software#XXXXXXXXX.com
always-auth=true
It's a private repo and the authTokens are currently hardcoded in the .npmrc file.
However while trying to find a solution for this, I did come across this random comment from a Github staff member: https://github.community/t/netlify-getting-401-from-github-package-registry-with-auth-token/16415/3
It's a bit vague, but it sounds like it doesn't accept a hardcoded authToken in the .npmrc file.
So first thing I tried was to use our env variable instead like so:
#xxxx=https://npm.pkg.github.com
//npm.pkg.github.com:_authToken=${NPM_AUTH_TOKEN}
The env variable is correct in our Github repo secrets, and supplied by the workflow.
However this still resulted in the same 401 Unauthorized error.
From looking at other solutions I then tried to generate the .npmrc manually inside the Github action before the install step, like so:
- name: npmrcgen
run: |
echo "//npm.pkg.github.com/:_authToken=XXXXXXX" > .npmrc
echo "#xxxxx=https://npm.pkg.github.com/" >> .npmrc
echo "#react-admin:registry=https://registry.marmelab.com" >> .npmrc
echo "//registry.marmelab.com:" >> .npmrc
echo "_auth=XXXXXXX" >> .npmrc
echo "email=software#xxxxx.com" >> .npmrc
echo "always-auth=true" >> .npmrc
During the logging step I added, it the _authToken (only for Github) still shows up as ***, and I still got a 401 Unauthorized error.
At this point I wanted to confirm the .npmrc was even being used, so I removed the second private registry we used for marmelab.com, and sure enough, I got an error saying it was no longer able to install their ra-realtime package. This proves the .npmrc file is indeed being read and used by my Github action, but it's not accepting my Github personal access token.
I have tried to generate a new token as well. It has full access to everything under repo: as well as write:packages and read:packages which is what should be required.
Still 401 Unauthorized in the Github action, and still works fine locally.
Lastly I have tried to install it with yarn instead of npm. Unsurprisingly this did not fix it either.
I have seen and tried the following solutions without any success:
Download private module from Github Package Registry via Yarn within a Github Action? Publishing works, but installing is met with '401 Unauthorized'
https://github.com/FerLuisxd/create-npmrc
https://blog.bitsrc.io/install-npm-private-packages-in-ci-cd-with-github-actions-746db95017cc
One thing I have not tried, as I have seen no recommendations on how or this being a good idea, but I have not done an npm login within the Github action. Since no one else has done this, and somehow have it working, I assume this is not necessary.
I contacted GitHub support and they managed to figure out what the problem was.
Github workflows are more strict than local environments and require an extra / before the auth token:
spot the difference:
//npm.pkg.github.com:_authToken=XXXXXXXXX. # broken
//npm.pkg.github.com/:_authToken=XXXXXXXXX # works
adding the extra / before :_authToken= solved the issue for me.
Have a .npmrc file in root of your project.
Content of .npmrc:
registry=https://registry.npmjs.org/
#{scope}:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=********** (Token generated from github)
#{scope} is your organization-name or your username. It is case-sensitive.
Addendum for anyone else who, like me, runs across this question outside the context of GitHub Actions: note that the GitHub package registry uses HTTP Basic Authentication. So if you're trying to test a personal access token and don't want to mess with your .npmrc / .yarnrc, you can pass the credentials in the registry URL, e.g. with yarn:
yarn info "#<github-org>/<repo-name>" \
--registry="https://<github-user>:<token>#npm.pkg.github.com/"
Or with curl:
curl -vL 'http://<github-user>:<token>#npm.pkg.github.com/#<github-org>%2f<repo-name>'
Just use actions/setup-node action.
- uses: actions/setup-node#v3
with:
node-version: 16
cache: "yarn"
registry-url: "https://npm.pkg.github.com"
- name: Build
env:
# also other environment variable
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
yarn
yarn build
This problem is happening due to Visibility access, to be able to delete packages you need Admin Permission.
Even when trying to do this by separate method without github Actions you still need Admin Permission for deletion.
even when you want to use other methods to Delete packages such as following code below, You still need Admin Permissions and PAT token to do so with delete:packages permissions.
I’ve written the kind of cleanup script I was thinking of here using the new packages delete/restore API . 🙂
I had a similar thought. 😃
Here’s a GitHub Action script that can be used to delete untagged images for a specified container package:
- uses: actions/github-script#v3
with:
github-token: ${{ secrets.DELETE_PACKAGES_TOKEN }}
script: |
const response = await github.request("GET /${{ env.OWNER }}/packages/container/${{ env.PACKAGE_NAME }}/versions",
{ per_page: ${{ env.PER_PAGE }}
});
for(version of response.data) {
if (version.metadata.container.tags.length == 0) {
console.log("delete " + version.id)
const deleteResponse = await github.request("DELETE /${{ env.OWNER }}/packages/container/${{ env.PACKAGE_NAME }}/versions/" + version.id, { });
console.log("status " + deleteResponse.status)
}
}
env:
OWNER: user # or orgs/<org name>
PACKAGE_NAME: <package name>
PER_PAGE: 100
OWNER should be either a user name or orgs/ORG_NAME.
DELETE_PACKAGES_TOKEN is a PAT with the delete:packages and write:packages scopes.

Concourse Slack alert

I have written a bash script it process some data and puts in one file. My intention is to give slack alert if there is content in that file if not it should not give the alert. Is there a way to do it? In Concourse
You should take advantage of the Concourse community's open source resource types. There's a list here. There is a slack resource listed on that page, but I use the one here (not included in the list above because it has not been added by the authors) https://github.com/cloudfoundry-community/slack-notification-resource.
That will give you the ability to add a put step in your job plan to send a slack resource. As for the logic of your original ask, you can use try and on_success. Your task might look something like this:
- try:
task: do-a-thing
config:
platform: linux
image_resource:
type: registry-image
source:
repository: YOUR_TASK_IMAGE
tag: latest
inputs:
- name: some-input
params:
FILE: some-file
run:
path: /bin/sh
args:
- -ec
- |
[ ! -z `cat some-input/${FILE}` ]
on_success:
put: slack
params:
<your slack resource put params go here>
The on_success part will run if the code defined in the task's run section returns 0. The script listed there just checks to see if there are more than zero bytes in the file. Because the task is wrapped in a try step, regardless of whether or not the task succeeds (and hence, sends you a message), the step will succeed and move to the next step in the plan.

Ansible uri module

I have web url and i have two options enable/disable , Now I am writing ansible playbook to do this which is in Powershell script
My ansible playbook
---
- hosts: localhost
vars:
applauncher: "xweb"
jobState: "Disable"
serverName: "NETBATCH"
jobName: "Loan1"
tasks:
- name: edit app jobs
uri:
url: 'http://{{ applauncher }}/Edit/{{ jobState }}?Server={{ serverName }}&JobName={{ jobName }}'
method: POST
user: xxxx
password: xxx
I am passing extra vars to disable and enable option , Is it possible to use this option in uri module and which method I should use to select disable/enable option in URL
I think this is what you are after, but it took me a while to figure out what you are asking for. Read about vars_prompt playbook keyword: https://docs.ansible.com/ansible/latest/user_guide/playbooks_prompts.html