Github Actions restored cache is not being used - github

I'm having hard time figuring out how to make the pipeline use the restored cache of npm modules.
Here's the manifest file:
jobs:
setup-build-push-deploy:
name: Setup, Build, Push, and Deploy
runs-on: ubuntu-latest
steps:
# Checkout the repo
- name: Checkout
uses: actions/checkout#v2
- name: Setup NodeJS
uses: actions/setup-node#v1
with:
node-version: '12.14'
# Cache the npm node modules
- name: Cache node modules
uses: actions/cache#v1
id: cache-node-modules
env:
CACHE_NAME: cache-node-modules
with:
# npm cache files are stored in `~/.npm` on Linux/macOS
path: ~/.npm
key: ${{ env.CACHE_NAME }}-${{ hashFiles('app/package-lock.json') }}
restore-keys: |
${{ env.CACHE_NAME }}-
# Install NPM dependencies
- name: Install Dependencies
if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: |
cd app/
npm install
# Compile Typescript to Javascript
- name: Compile Typescript
run: |
cd app/
npm run make
The "cache" step does make a successful cache hit, and therefore the "install dependencies" step is skipped. But then the "compile" step fails, because it cannot find any installed npm modules.
The docs here and here and the SO question, eg. here, do not specify any certain step or configuration that points towards where to pick up the cached modules from. Seems like they should be picked up automatically from ~/.npm, but that's not the case.

I was able to solve this by changing the path to node_modules:
Here's an example:
jobs:
prepare-npm-cache:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: 12.22.0
- name: Cache node modules
uses: actions/cache#v2
id: npm_cache_id
with:
path: node_modules
key: ${{ runner.os }}-npm-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-npm-cache-
${{ runner.os }}-
- name: Install Dependencies
if: steps.npm_cache_id.outputs.cache-hit != 'true'
run: npm ci

I found myself in the exact same situation.
The node_modules seems to be cached but then the next job immediately fails due to missing dependencies.
I ended up using this GitHub action by Gleb Bahmutov.
Specifically this part:
- uses: bahmutov/npm-install#v1.4.5
- run: npm install
- run: npm run ***
Detailed description can be found in this article:
https://glebbahmutov.com/blog/do-not-let-npm-cache-snowball/

Related

how to run GitHub Action after outage?

As you may (or may not) know yesterday was a major incident of GitHub's services: https://www.githubstatus.com/incidents/tyc8wpsgr2r8.
Unfortunately I published a release during that time and the action responsible for building and publishing the code didn't trigger.
For actions which were executed at least once I have an option to "Re-run workflow" - but how can I proceed with an action which didn't even trigger - I can not see it anywhere whatsoever?
I think the last resort would be to just make another release, remove the problematic one etc. but I'd like to avoid that.
The workflow file:
name: Node.js CI
on:
push:
branches: [master]
release:
types: [published]
pull_request:
branches: [master]
jobs:
test:
name: Test Node.js v${{ matrix.node-version }}
runs-on: ubuntu-latest
strategy:
matrix:
node-version:
- 16
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
- run: npm install --production=false --no-package-lock
- name: Lint 💅🏻
run: npm run lint
- run: npm test
release:
name: Publish NPM Package
if: startsWith(github.ref, 'refs/tags/')
needs:
- test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: 16
registry-url: 'https://registry.npmjs.org'
- run: npm install --production=false --no-package-lock
- run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
gh-pages:
name: Publish GitHub Pages
if: ${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main' }}
needs:
- test
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: 16
registry-url: 'https://registry.npmjs.org'
- name: Install ✔️
run: npm install --production=false --no-package-lock
- name: Build storybook 🏗️
run: npm run build-storybook
- name: Deploy 🚀
uses: JamesIves/github-pages-deploy-action#4.1.3
with:
branch: gh-pages
folder: storybook-static
As you said in the comment, the easiest solution would be to remove the release and create it all over again.
Another option could be to add a workflow_dispatch event trigger to the workflow with a tag input, updating the jobs condition to use this input.tag variable if informed.
That way, if an automatic trigger failed (through push, release or pull_request), you could trigger it manually through the Github UI or the GH CLI as an alternative.

Cache implementation not working in github action

I've tried to follow the official documentation, several blogs and answers on stackoverflow but nothing helped me to find a solution for this problem.
With github-action I've created a script to run unit tests after every push is done in our github server. For now the project is in early stage, so our dependency list is quite short and npm install command is done quite quickly (around 40 seconds). Obviously in future we'll need to add more dependencies and we are expecting a serious increment of that time, so I've tried to implement the cache mechanism, but for some reason the process still need to download the dependencies and the time for that step still be pretty much the same.
Below there is the script I've used. Following the documentation online I've found 2 different options, I've tried both (once per time) but without luck.
name: unit
on:
push:
branches: ['*']
pull_request:
branches: ['*']
workflow_dispatch:
jobs:
build:
runs-on: macos-latest
steps:
- uses: actions/checkout#v2
- name: setup node
uses: actions/setup-node#v2
with:
node-version: '15'
cache: npm
# start option 1
- name: Setup cache
uses: actions/cache#v2
with:
path: app/node_modules
key: ${{ runner.os }}-node-${{ hashFiles('app/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
# end option 1
# start option 2
- name: Cache node modules
uses: actions/cache#v2
env:
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
# end option 2
- name: Install dependencies
run: npm install
- name: run unit tests
run: npm run test
Why the cache is not working? Am I doing some mistake in the script?

How to cache npm dependencies in GitHub action?

Below is my dockerfile. Is there a way to cache npm in GitHub action?
FROM node
WORKDIR /app
ADD package*.json ./
RUN npm ci
ENV PATH /app/node_modules/.bin:$PATH
My GitHub actions:
name: NPM buid
on:
push:
branches:
- main
jobs:
build-npm-image:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Build and Tag Node image
id: build-ui-image
run: |
export DOCKER_BUILDKIT=1
docker build -t ui -f ./ui/Dockerfile .
P.S I don't want to cache docker image. Above docker file is used just for example. objection is cache npm dependencies
This can be achieved with actions/setup-node#v3
steps:
- uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: current
cache: npm
- name: Build and Tag Node image
id: build-ui-image
run: |
export DOCKER_BUILDKIT=1
docker build -t ui -f ./ui/Dockerfile .
Did you check below action ?
actions/cache#v2
So your Actions yaml would look like below,
name: NPM buid
on:
push:
branches:
- main
jobs:
build-npm-image:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- uses: actions/setup-node#v2
with:
node-version: '14'
- name: Caching node modules
uses: actions/cache#v2
with:
path: "app/node_modules"
key: node-modules-${{ hashFiles('app/package.json') }}
- name: Build and Tag Node image
id: build-ui-image
run: |
export DOCKER_BUILDKIT=1
docker build -t ui -f ./ui/Dockerfile .
Caching npm dependencies will not make a difference if you're building a docker image. You can use actions/setup-node for caching JS or docker/build-push-action for Docker.

Github actions how to do npm install only 1 time

I'm using GitHub actions for deploy. And i need to build vuejs app, every time when pushing. Now all working coorectly, but every time i spend action minutes for npm install.
Can i install modules 1 time, and not reinstall it after every deploy?
on: push
name: 🚀 Deploy website on push
jobs:
web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [15.x]
steps:
- name: 🚚 Get latest code
uses: actions/checkout#v2.3.2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- run: npm install --legacy-peer-deps
- run: npm run build
- name: 📂 Sync files
uses: SamKirkland/FTP-Deploy-Action#4.0.0
with:
server: ${{ secrets.host}}
username: ${{ secrets.user}}
password: ${{ secrets.pass }}
local-dir: dist/
dangerous-clean-slate: true

Github CI/CD pipeline crashes with webpack

I'm trying to learn how CI/CD pipelines work.
I decided to use it with my portfolio page which should re-run itself on every push.
Here is my yaml config:
name: Build Bundle for Github Pages
on:
push:
branches:
- source
env:
NODE_ENV: production
PUBLIC_URL: http://crrmacarse.github.io/
GA_TRACKING_CODE: ${{ secrets.GA_TRACKING_CODE }}
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v1
with:
persist-credentials: false
- name: Build
run: |
npm install
npm run prod:pipeline
npm run sitemap
cp dist/index.html dist/404.html
cp google21029c74dc702d92.html dist/
cp robots.txt dist/
- name: Deploy
uses: JamesIves/github-pages-deploy-action#releases/v3
with:
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
BRANCH: source
FOLDER: dist
Here is the error:
Source code for the webpack config:
https://github.com/crrmacarse/crrmacarse.github.io/blob/source/compiler/production.pipeline.js
If the error message is cannot find module 'html-webpack-plugin', you could try, for testing, to install it.
See survivejs/webpack-book issue 100 as an example:
The solution was to run npm i html-webpack-plugin --save-dev before building with webpack
The OP has fixed the GitHub Action workflow with crrmacarse/crrmacarse.github.io commit 8a4397b
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2.0.0
- name: Use node 12
uses: actions/setup-node#v1
with:
node-version: 12
registry-url: https://registry.npmjs.org
- name: install
run: npm install
- name: lint
run: npm run sitemap
- name: build
run: npm run prod
- name: copy
run: npm run copy
Then the module html-webpack-plugin is properly installed and available, as seen in this Actions CI run.
Use npm install before build your project which installs your npm library on docker where your project will be built.
Note: Don't forget to define html-webpack-plugin on packge.json file