I am building a CI workflow using GitHub Actions.
Goal is to build and test a .NET C# application using a MSSQL database.
The database can successfully start and the database is created. In the logs of the DB container I can see the DB was created and is running.
In the testing step all tests fail with this error:
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Resource temporarily unavailable. Aborting test execution.
The database name is correctly given to the program. I have tested this by printing the connection string to the console. Also it actually connects to the DB (because when the database server name is incorrent I get a error that reflects that).
The CI workflow:
name: .NET Backend Build and run Unit Tests
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build_and_test:
runs-on: ubuntu-latest
services:
sqlserver:
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- 1433:1433
env:
ACCEPT_EULA: "Y"
SA_PASSWORD: "redacted"
MSSQL_PID: "Express"
MSSQL_COLLATION: "SQL_Latin1_General_CP1_CI_AS"
steps:
- name: get Container ID
run: echo "DATABASE_SERVER=$(docker ps --all --filter status=running --format "{{.ID}}")" >> $GITHUB_ENV
- name: create database
run: docker exec $DATABASE_SERVER /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'redacted' -Q 'CREATE DATABASE dbname'
- uses: actions/checkout#v3
- name: Setup .NET
uses: actions/setup-dotnet#v3
with:
dotnet-version: 6.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore
- name: Test
env:
DATABASE_PORT: 1433
DATABASE_NAME: dbname
DATABASE_USER: sa
DATABASE_PASSWORD: redacted
run: dotnet test --no-build --verbosity normal
I found out the Docker in GitHub Actions does not support DNS and behaves fundamentally different than regular Docker on Linux. All network communication must be made over the host network. So I had to specify localhost as the database server.
This would be the correct workflow file:
name: .NET Backend Build and run Unit Tests
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
env:
CI: true
jobs:
build_and_test:
runs-on: ubuntu-latest
services:
sqlserver:
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- 1433:1433
env:
ACCEPT_EULA: "Y"
MSSQL_SA_PASSWORD: "redacted"
MSSQL_PID: "Express"
MSSQL_COLLATION: "SQL_Latin1_General_CP1_CI_AS"
steps:
- name: get Container ID
run: echo "DATABASE_ID=$(docker ps --all --filter status=running --format "{{.ID}}")" >> $GITHUB_ENV
- name: create database
run: docker exec $DATABASE_ID /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'redacted' -Q 'CREATE DATABASE dbname'
- uses: actions/checkout#v3
- name: Setup .NET
uses: actions/setup-dotnet#v3
with:
dotnet-version: 6.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore
- name: Test
env:
DATABASE_SERVER: localhost
DATABASE_PORT: 1433
DATABASE_NAME: dbname
DATABASE_USER: sa
DATABASE_PASSWORD: redacted
run: dotnet test --no-build --verbosity normal
Related
I am trying to build an action that is triggered on creating a new release on GitHub which works fine, but I would like to reference the tag in my action:
name: Build production container
on:
release:
types:
- created
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Build the Docker image
run: |
echo "${{ SECRET }}" | docker login -u ME --password-stdin docker.pkg.github.com
docker build app/ -t docker.pkg.github.com/REPO_PATH/image:$VERSION
docker push docker.pkg.github.com/REPO_PATH/image:$VERSION
shell: bash
env:
VERSION: 0.0.1
This is my working action, but I would like to automatically pull the tag into the VERSION environment variable. I read the documentation, especially here where the GitHub context is referenced, but I can't seem to find anything about it.
It took me a while to figure out that the action has a different context for each method documented here. So the parameter I was looking for is the and I've set my action up after this example:
name: Build production container
on:
release:
types:
- created
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Get Tag Name
id: tag_name
run: |
echo ::set-output name=SOURCE_TAG::${GITHUB_REF#refs/tags/}
- name: Build the Docker image
run: |
echo "${{ SECRET }}" | docker login -u ME --password-stdin docker.pkg.github.com
docker build app/ -t docker.pkg.github.com/REPO_PATH/image:$VERSION
docker push docker.pkg.github.com/REPO_PATH/image:$VERSION
shell: bash
env:
VERSION: ${{ steps.tag_name.outputs.SOURCE_TAG }}
This basically adds getting the source parameter as an extra step, this way I can use it in the environment variables of the next step.
I want to run a script in the postgres service in github actions that creates a table and adds an extension. How can I do that? Do I need to make a shell script or can I do right in the yaml file?
sql script
drop database mydb;
create database mydb;
\c mydb;
CREATE EXTENSION "pgcrypto";
workflow
name: API Integration Tests
on:
pull_request:
push:
branches:
-master
env:
DB_HOST: localhost
DB_USERNAME: postgres
DB_PASSWORD: rt
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [10.x, 12.x, 13.x]
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: mydb
POSTGRES_PASSWORD: helloworl
POSTGRES_USER: postgres
ports:
- 5433:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout#v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install
run: npm ci
- name: npm test
run: npm run test
You can add a step that uses PSQL commands.
Here's an example step that creates your database:
- name: Create database
run: |
PGPASSWORD=helloworl psql -U postgres -tc "SELECT 'CREATE DATABASE mydb' WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'mydb')"
By the way, I note that the next command you wanted was: CREATE EXTENSION "pgcrypto";, which I assume is because you want to generate UUIDs (Common use case). Please note that you do not need this for get_random_uuid() as this is natively support in Postgres from v13 onwards.
However if you really, really, really wanted to add pgcrypto, you can use this step:
- name: Enable pgcrypto extension
run: |
PGPASSWORD=helloworl psql -U postgres -tc "CREATE EXTENSION 'pgcrypto';"
on github action after deploying prisma I want to seed the database. but this gives me project not found error. why this error is showing even after prisma is deployed?
this is my yaml file
name: "Github Actions Test"
on:
push:
branches:
- wip/checkout2
jobs:
test:
runs-on: ubuntu-latest
env:
PRISMA_ENDPOINT: ${{secrets.PRISMA_ENDPOINT}}
PRISMA_SECRET: ${{secrets.PRISMA_SECRET}}
steps:
- uses: actions/checkout#v1
- name: "Install Node"
uses: actions/setup-node#v1
with:
node-version: "12.x"
- name: "Install global packages"
run: npm install -g yarn prisma-cli concurrently mocha wait-port
- name: "Run docker Container"
run: docker-compose -f docker-compose.yml up --build -d
- name: "Install deps"
run: yarn install
- name: "prisma deploy"
run: yarn deploy:backend
- name: "Seed Backend"
run: yarn seed:backend
- name: "Build app"
run: yarn build
- name: "Start backend and frontend concurrently on background and run tests"
run: |
yarn start &
yarn test
The following is my actions file.
name: ZAP
on: push
jobs:
build:
runs-on: ubuntu-latest
container:
image: owasp/zap2docker-stable
options: --user root
volumes:
- /__w/actions-test-repo/actions-test-repo:/zap/wrk/
steps:
- uses: actions/checkout#v1
- name: view file
run: pwd
- name: run zap
if: always()
run: zap-baseline.py -t https://www.example.com -g gen.conf -r testreport.html
- name: view file
if: always()
run: pwd
I want to bind the directory /zap/wrk/ to a local directory. But when the container starts it does not mount this volume. I got the present working directory and mounted it to the docker container. Is this the correct way to do it?
Results link: https://github.com/sshniro/actions-test-repo/commit/08c0257d92b772a1d33c0b68cb8af99afdef9130/checks?check_suite_id=324032091
Similar issues have been identified in this forum as well.
https://github.community/t5/GitHub-Actions/Container-volumes-key-not-mounting-volume/td-p/34798
The workaround is to use the options parameter.
name: ZAP
on: push
jobs:
build:
runs-on: ubuntu-latest
container:
image: owasp/zap2docker-stable
options: -v /__w/actions-test-repo/actions-test-repo:/zap/wrk/:rw
steps:
- uses: actions/checkout#v2
- name: run zap
if: always()
run: zap-baseline.py -t https://www.example.com -g gen.conf -w testreport.md
I am trying GitHub Actions for CI with a Ruby on Rails application.
My setup is with VM, not running the Ruby build in a container.
This is my workflow yml. It runs all the way without errors until the step "Setup Database".
name: Rails CI
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:10.10
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: db_test
ports:
- 5432/tcp
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:latest
ports:
- 6379/tcp
steps:
- uses: actions/checkout#v1
- name: Set up ruby 2.5
uses: actions/setup-ruby#v1
with:
ruby-version: 2.5.5
- name: Set up node 8.14
uses: actions/setup-node#v1
with:
node-version: '8.14'
- name: Setup system dependencies
run: sudo apt-get install libpq-dev
- name: Setup App Dependencies
run: |
gem install bundler -v 1.17.3 --no-document
bundle install --jobs 4 --retry 3
npm install
npm install -g yarn
- name: Run rubocop
run: bundle exec rubocop
- name: Run brakeman
run: bundle exec brakeman
- name: Setup Database
env:
RAILS_ENV: test
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
run: |
cp config/database.yml.ci config/database.yml
bundle exec rails db:create
bundle exec rails db:schema:load
- name: Run rspec
env:
RAILS_ENV: test
REDIS_HOST: redis
REDIS_PORT: ${{ job.services.redis.ports[6379] }}
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
run: bundle exec rspec --tag ~type:system
I am able to install ruby, node, the images, Postgres as a service, etc, and run Rubocop and Brakeman. But when I try to set up the DB before running Rspec it says it cannot connect to the DB.
As far as I've been able to ascertain, the host is localhost when running the VM configuration as opposed to a container configuration.
This is the database.yml.ci that the "Setup Database" step copies to the database.yml to be used by Rails.
test:
adapter: postgresql
encoding: unicode
database: db_test
pool: 5
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
host: <%= ENV['POSTGRES_HOST'] %>
I expected Postgres to be correctly set up and bundle exec rails db:create to create the database. However, it throws the following error:
rails aborted!
PG::ConnectionBad: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
I've tried all sorts of different configurations, but unfortunately, Actions is sort of knew and there doesn't seem to be a lot of material available online.
Any ideas on how to fix this?
===========================
EDIT:
So I was able to sort this out through trial and error. I ended up using a docker image with a ruby and node container. This is the working configuration:
on:
push:
branches:
- master
pull_request:
branches:
- master
- development
- release
jobs:
build:
runs-on: ubuntu-latest
container:
image: timbru31/ruby-node:latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ci_db_test
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
chrome:
image: selenium/standalone-chrome:latest
ports:
- 4444:4444
steps:
- uses: actions/checkout#v1
- name: Setup app dependencies
run: |
gem install bundler -v 1.17.3 --no-document
bundle install --jobs 4 --retry 3
npm install
npm install -g yarn
- name: Run rubocop
run: bundle exec rubocop
- name: Run brakeman
run: bundle exec brakeman
- name: Setup database
env:
RAILS_ENV: test
POSTGRES_HOST: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ci_db_test
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
run: |
cp config/database.yml.ci config/database.yml
bundle exec rails db:create
bundle exec rails db:schema:load
- name: Run rspec
env:
RAILS_ENV: test
POSTGRES_HOST: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ci_db_test
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
SELENIUM_URL: 'http://chrome:4444/wd/hub'
run: bundle exec rspec
And the CI DB configuration database.yml.ci
default: &default
adapter: postgresql
encoding: unicode
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
host: <%= ENV['POSTGRES_HOST'] %>
pool: 5
database: <%= ENV['POSTGRES_DB'] %>
test:
<<: *default
I have a slightly different setup but this was the most relevant question when I encountered the same error so wanted to post here in case it can help. The two things that were critical for me were:
1) Set the DB_HOST=localhost
2) Set the --network="host" argument when you start the docker container with your rails app
name: Master Build
on: [push]
env:
registry: my_registry_name
# Not sure these are actually being passed down to rails, set them as the default in database.yml
DB_HOST: localhost
DB_USERNAME: postgres
DB_PASSWORD: postgres
jobs:
my_image_test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Check out repository
uses: actions/checkout#v2
- name: Build my_image docker image
uses: whoan/docker-build-with-cache-action#v5
with:
username: "${{secrets.aws_ecr_access_key_id}}"
password: "${{secrets.aws_ecr_secret_access_key}}"
registry: "${{env.registry}}"
image_name: my_image
context: my_image
- name: Lint rubocop
working-directory: ./my_image
run: docker run $registry/my_image bundle exec rubocop
- name: Run rails tests
working-directory: ./my_image
run: docker run --network="host" $registry/my_image bash -c "RAILS_ENV=test rails db:create && RAILS_ENV=test rails db:migrate && rails test"
Your problem appears to be that Postgres is not exposed on port 5432. Try to replace the port number with ${{ job.services.postgres.ports[5432] }}.
There are examples here: https://github.com/actions/example-services/blob/master/.github/workflows/postgres-service.yml
I had this challenge when trying to set up GitHub actions for a Rails Application.
Here's what worked for me:
name: Ruby
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
ruby-version:
- '2.7.2'
node-version:
- '12.22'
database-name:
- my-app
database-password:
- postgres
database-user:
- postgres
database-host:
- 127.0.0.1
database-port:
- 5432
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: ${{ matrix.database-name }}
POSTGRES_USER: ${{ matrix.database-user }}
POSTGRES_PASSWORD: ${{ matrix.database-password }}
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options:
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Check out Git Repository
uses: actions/checkout#v2
- name: Set up Ruby, Bundler and Rails
uses: ruby/setup-ruby#v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
- name: Set up Node
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: Install packages
run: |
yarn install --check-files
- name: Setup test database
env:
RAILS_ENV: test
DATABASE_NAME_TEST: ${{ matrix.database-name }}
DATABASE_USER: ${{ matrix.database-user }}
DATABASE_PASSWORD: ${{ matrix.database-password }}
DATABASE_HOST: ${{ matrix.database-host }}
DATABASE_PORT: ${{ matrix.database-port }}
POSTGRES_DB: ${{ matrix.database-name }}
run: |
bundle exec rails db:migrate
bundle exec rails db:seed
Note:
Replace my-app with the name of your app.
You can leave the database-password and database-user as postgres
That's all.
I hope this helps