I am trying to use Github actions on the Push event to my master branch for running a wget command to mirror a website and download its contents as static files and then zip them together for uploading to an s3 bucket. Here is my test_events.yml file stored under .github/workflows in my git repo:
name: create a mirror of website and zip the saved contents
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
mirror-website:
name: mirrors the website
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v2
- name: wget
uses: wei/wget#v1
with:
args: -O logfile.txt -P ./actions/ https://adappt.co.uk
zip-files:
name: Zips the Saved file
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v2
- name: Prepares a zip file
uses: montudor/action-zip#v1.0.0
with:
args: zip -qq -r adappt_co_uk.zip . -i /actions/adappt.co.uk
Problem is whenever I push a change in my repo, the action gets triggered and gets completed without errors. But it does not save the web files to the given location, and hence, doesn't perform the zip operation as well.
What am I doing wrong here? Is there any other way apart from using wget?
Related
I have added workflow to github actions where i want to rsync files from repository to remote server
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Deploy with rsync
run: rsync -avuz --delete . ${{ secrets.USER }}#${{ secrets.HOST }}:~/App/MainPage/
rsync sends nothing because my repository in source folder is empty, even though there are files in repo.
i get this message
I know that rsync works, because if i change dir (eg ../../) files are being transferred to remote server and i can see them there.
I forgot to add checkout action to workflow .yaml file
...
jobs:
build:
runs-on: ubuntu-latest
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v3
- name: Install SSH Key
uses: shimataro/ssh-key-action#v2
....
I am trying to use pandoc to convert markdown files pushed to my repo into pdfs. But, not sure how to save the output to my repo.
Here's my code:
name: pandoc
on: push
jobs:
convert_via_pandoc:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout#v2
- name: convert md to pdf
uses: docker://pandoc/latex:2.9
with:
entrypoint: /bin/sh
run: |
cd markdown-files;
for file in $(ls ./ |grep *.md); do
pandoc $file -o ${file:0:-2}pdf;
done
You can simply push it back to the repository by committing it with one of the actions available on Marketplace:
- uses: stefanzweifel/git-auto-commit-action#v4
with:
commit_message: Changed files
Alternatively, you can upload your data as an artifact to your workflow:
https://github.com/actions/upload-artifact
- uses: actions/upload-artifact#v3
with:
name: pdfs
path: path/to/pdfs/
I´m tring to config a GitHub Action to deploy my application to SFTP file.
My application has 6700 files and I would like to upload only changed/commited files.
How can I remove unchanged and/or uncommited files before upload to SFTP?
This way, my one file modification deploy would be so faster than upload 6k files.
name: CI
on:
push:
branches: [ main ]
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-latest
name: Deploy Job
steps:
- name: Checkout
uses: actions/checkout#v2
with:
fetch-depth: 2
- name: Deploy files
uses: wlixcc/SFTP-Deploy-Action#v1.0
with:
username: 'deploy_user'
server: 'server_ip'
ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }}
local_path: './www/*'
remote_path: '/www'
args: '-o ConnectTimeout=10'
To list down all the files that are updated/committed in the given commit, you can use this command:
$ git diff-tree --no-commit-id --name-only -r $GITHUB_SHA
index.html
src/application.js
Then you can use this to delete all the files that are not on this list. This needs some bash digging. One hack I can think of is to make a temporary directory to copy updated files and only upload these files.
How can I push some files that were generated by the runner (user1/repo1) to the main branch from another remote repo (user2/repo1) via GitHub actions?
Please note that:
I set-up a secret key (named ACCESS_TOKEN) in user1/repo1, such that it corresponds to the Personal Access Token from the destination repo (user2/repo1)
the GitHub actions needs to be repeated every ~30 minutes
there already exists a file.rds in the destination repo. The push thus needs to override that file every time
the runner needs to be macOS-latest
This is what I have tried so far:
name: gitaction
on:
schedule:
- cron: "*/30 * * * *"
workflow_dispatch:
jobs:
genFileAndPush:
runs-on: macOS-latest
steps:
- uses: actions/checkout#master
- uses: r-lib/actions/setup-r#master
with:
r-version: '4.1.2'
- name: Run R scripts and generate file
run: |
saveRDS(1:3, file = "file.rds")
shell: Rscript {0}
- name: Push to remote repository
run: |
git config --local user.name actions-user
git config --local user.email "actions#github.com"
git add file.rds
git commit -m "commit"
git remote set-url origin https://env.REPO_KEY#github.com/user2/repo1.git
git push -u origin main
env:
REPO_KEY: ${{secrets.ACCESS_TOKEN}}
username: github-actions
It returns the following error:
remote: Permission to user2/repo1.git denied to github-actions[bot].
fatal: unable to access 'https://github.com/user2/repo1.git/': The requested URL returned error: 403
Error: Process completed with exit code 128.
What am I missing?
Edit
As suggested, I tried using GuillaumeFalourd/git-commit-push#v1.1:
name: gitaction
on:
workflow_dispatch:
jobs:
genFileAndPush:
runs-on: macOS-latest
steps:
- uses: actions/checkout#master
- uses: r-lib/actions/setup-r#master
with:
r-version: '4.1.2'
- name: Run R scripts and generate file
run: |
saveRDS(1:3, file = "file.rds")
shell: Rscript {0}
- uses: actions/checkout#v2.3.4
- uses: GuillaumeFalourd/git-commit-push#v1.1
with:
target_branch: main
files: file.rds
remote_repository: https://github.com/user2/repo1
access_token: ${{secrets.ACCESS_TOKEN}}
force: true
Although there were no error, the file was not pushed (because it was not detected?):
Run GuillaumeFalourd/git-commit-push#v1.1
Run CURRENT_BRANCH=${GITHUB_REF}
WARNING: No changes were detected. git commit push action aborted.
There are some actions on the Github Marketplace that can help you with pushing files to other repositories.
Here is an example of one supported on all OS runners.
The workflow would look like this:
name: gitaction
on:
workflow_dispatch:
jobs:
genFileAndPush:
runs-on: macOS-latest
steps:
- uses: actions/checkout#master
- uses: r-lib/actions/setup-r#master
with:
r-version: '4.1.2'
- name: Run R scripts and generate file
run: |
saveRDS(1:3, file = "file.rds")
shell: Rscript {0}
- uses: GuillaumeFalourd/git-commit-push#v1.3
with:
target_branch: main
files: file.rds
remote_repository: https://github.com/user2/repo1
access_token: ${{secrets.ACCESS_TOKEN}}
force: true
You can find more actions like this one on the marketplace.
Otherwise, you can also perform the whole operation manually using command lines to clone the remote repository, copy the files from the local repo wherever you want on the remote repo, then push the new files to the remote repository.
I am trying to upload a repo to server via ftp on push to master branch. I have it set up and working. However in the repo there is a folder /public. I only want to upload the files in this folder to the server. Not other files or the folder itself. I have tried to set up a working directory for the job but this doesn't seem to do the trick.. any ideas?
on:
push:
branches:
- master
name: 🚀 Deploy website on push
jobs:
ftp-web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./public
steps:
- name: 🚚 Get latest code
uses: actions/checkout#v2.4.0
working-directory: ./public
with:
fetch-depth: 2
- name: 📂 Sync files
uses: SamKirkland/FTP-Deploy-Action#4.2.0
with:
server: ****
username: ****
password: ${{ secrets.prod_ftp_password }}
server-dir: public_html/
Checking out only one directory is not possible, but has been requested in the actions/checkout repository before: https://github.com/actions/checkout/issues/483
There's an action to check out specific files, but I haven't tried it and I'm not sure if it does what you want: https://github.com/marketplace/actions/checkout-files
You might want to ask yourself why you're trying to limit the number of files transferred. Is it because you're concerned about traffic? Or because of the input expected in the subsequent action?
If it's the latter, you could also manually "fix" the structure by running some mv and rm commands.