Can github actions include postgresql with uuid support? - postgresql

I have a working GitHub action which installs PostgreSQL 11. But now I use UUIDs and those are not supported.
I need to run CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; to install UUIDs but it is not clear how to do that with GitHub Actions.
I've thrashed and tried several other Docker images which have UUID support enabled but they are old user throw-always and do not support Actions.
My Rust.yml is below:
name: Rust
on: [push]
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
rust: [stable, beta]
services:
postgres:
image: postgres:11.6
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
env:
RUSTFLAGS: -D warnings
CARGO_INCREMENTAL: 0
RUN_SLOW_TESTS: 1
RUSTUP_MAX_RETRIES: 10
CARGO_NET_RETRY: 10
steps:
- uses: hecrj/setup-rust-action#v1
with:
rust-version: ${{ matrix.rust }}
components: rustfmt
targets: wasm32-unknown-unknown
- uses: actions/checkout#master
- name: Install Dependencies
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get update && sudo apt-get install libudev-dev zlib1g-dev alsa libasound2-dev
- name: Build
run: cargo build --verbose
- name: Install Diesel CLI
run: cargo install diesel_cli --no-default-features --features postgres
- name: Setup Diesel
env:
DATABASE_URL: postgres://postgres:postgres#localhost/nof1_time_series
run: diesel setup
- name: Run tests
env:
DATABASE_URL: postgres://postgres:postgres#localhost/nof1_time_series
run: cargo test --verbose

The solution was to include the following /migrations/00000000000010_install_uuid_feature/Up.sql as an own migration early in the migration set and run as superuser postgres:
-- Install the UUID extension to this database
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
Other UUID tips if you are using Rust Diesel as your ORM:
Make sure to include the Diesel features you need:
[dependencies]
diesel = { version = "1.4", features = ["postgres", "chrono", "uuidv07", "serde_json"] }
uuid = {version = "0.8", features = ["serde", "v4"]}
chrono = { version = "0.4", features = ["serde"] }
Do not check formatting in the Github Action- the generated Diesel Schema will cause that to fail the build.
My updated Github Action for Rust Diesel with UUID support is below:
name: Rust
on: [push]
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
rust: [stable, beta]
services:
postgres:
image: postgres:11.6
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
env:
RUSTFLAGS: -D warnings
CARGO_INCREMENTAL: 0
RUN_SLOW_TESTS: 1
RUSTUP_MAX_RETRIES: 10
CARGO_NET_RETRY: 10
steps:
- uses: hecrj/setup-rust-action#v1
with:
rust-version: ${{ matrix.rust }}
components: rustfmt
targets: wasm32-unknown-unknown
- uses: actions/checkout#master
- name: Install Dependencies
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get update && sudo apt-get install libudev-dev zlib1g-dev alsa libasound2-dev
- name: Build
run: cargo build --verbose
- name: Install Diesel CLI
run: cargo install diesel_cli --no-default-features --features postgres
- name: Setup Diesel
env:
DATABASE_URL: postgres://postgres:postgres#localhost/timeseries
run: diesel setup
- name: Run tests
env:
DATABASE_URL: postgres://postgres:postgres#localhost/timeseries
run: cargo test --verbose
Complete example time-series-database

Related

stuck on "starting your workflow run" on Github Actions

I wanted to test "github action" feature but it is not starting and its is stuck.It just says "Starting your workflow run..."
Is there something wrong in my build.yml file
This is my build.yml file:
name: CI
on:
pull_request:
branches:
- master
workflow_dispatch:
env:
POSTGRESQL_VERSION: 13.1
POSTGRESQL_DB: students_info
POSTGRESQL_USER: postgres
POSTGRESQL_PASSWORD: password
JAVA_VERSION: 1.15
jobs:
build:
runs-on: ubuntu-16.04
services:
postgres:
image: postgres:13.1
env:
POSTGRES_DB: ${{ env.POSTGRESQL_DB }}
POSTGRES_USER: ${{ env.POSTGRESQL_USER }}
POSTGRES_PASSWORD: ${{ env.POSTGRESQL_PASSWORD }}
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout#v2
- uses: actions/setup-java#v1.4.3
with:
java-version: ${{ env.JAVA_VERSION }}
- name: Maven Clean Package
run: |
./mvnw --no-transfer-progress clean package -P build-frontend
PS:I have tried with ubuntu-latest as well
There is currently a problem with GitHub Actions:
https://www.githubstatus.com/incidents/zbpwygxwb3gw

github actions - run sql script in postgres service

I want to run a script in the postgres service in github actions that creates a table and adds an extension. How can I do that? Do I need to make a shell script or can I do right in the yaml file?
sql script
drop database mydb;
create database mydb;
\c mydb;
CREATE EXTENSION "pgcrypto";
workflow
name: API Integration Tests
on:
pull_request:
push:
branches:
-master
env:
DB_HOST: localhost
DB_USERNAME: postgres
DB_PASSWORD: rt
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [10.x, 12.x, 13.x]
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: mydb
POSTGRES_PASSWORD: helloworl
POSTGRES_USER: postgres
ports:
- 5433:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout#v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install
run: npm ci
- name: npm test
run: npm run test
You can add a step that uses PSQL commands.
Here's an example step that creates your database:
- name: Create database
run: |
PGPASSWORD=helloworl psql -U postgres -tc "SELECT 'CREATE DATABASE mydb' WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'mydb')"
By the way, I note that the next command you wanted was: CREATE EXTENSION "pgcrypto";, which I assume is because you want to generate UUIDs (Common use case). Please note that you do not need this for get_random_uuid() as this is natively support in Postgres from v13 onwards.
However if you really, really, really wanted to add pgcrypto, you can use this step:
- name: Enable pgcrypto extension
run: |
PGPASSWORD=helloworl psql -U postgres -tc "CREATE EXTENSION 'pgcrypto';"

Github Action: [!] Error: Cannot find module 'rollup-plugin-commonjs'

In my package.json there are rollup and rollup-plugin-commonjs
but inside github actions it could not find those packages!
If I do not add rollup in global package installation step of github-action it shows that rollup is not found. But after adding both rollup and rollup-plugin-commonjs I get [!] Error: Cannot find module 'rollup-plugin-commonjs'
this is my workflow file:
name: Github Action
on:
push:
branches:
- fix/auto-test
jobs:
test:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v1
- name: Bootstrap app on Ubuntu
uses: actions/setup-node#v1
with:
node-version: '11.x.x'
- name: Install global packages
run: npm install -g prisma rollup rollup-plugin-commonjs
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: Cache Project dependencies test
uses: actions/cache#v1
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install project deps
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn
- name: Run docker
run: docker-compose -f docker-compose.test.prisma.yml up --build -d
- name: Sleep
uses: jakejarvis/wait-action#master
with:
time: '30s'
- name: Reset the database for safety
run: yarn reset:backend
- name: Deploy
run: yarn deploy:backend
- name: Build this great app
run: yarn build
- name: start app and worker concurrently and create some instances
run: |
yarn start &
yarn start:worker &
xvfb-run --auto-servernum yarn test:minimal:runner

How to connect to Postgres in GitHub Actions

I am trying GitHub Actions for CI with a Ruby on Rails application.
My setup is with VM, not running the Ruby build in a container.
This is my workflow yml. It runs all the way without errors until the step "Setup Database".
name: Rails CI
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:10.10
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: db_test
ports:
- 5432/tcp
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:latest
ports:
- 6379/tcp
steps:
- uses: actions/checkout#v1
- name: Set up ruby 2.5
uses: actions/setup-ruby#v1
with:
ruby-version: 2.5.5
- name: Set up node 8.14
uses: actions/setup-node#v1
with:
node-version: '8.14'
- name: Setup system dependencies
run: sudo apt-get install libpq-dev
- name: Setup App Dependencies
run: |
gem install bundler -v 1.17.3 --no-document
bundle install --jobs 4 --retry 3
npm install
npm install -g yarn
- name: Run rubocop
run: bundle exec rubocop
- name: Run brakeman
run: bundle exec brakeman
- name: Setup Database
env:
RAILS_ENV: test
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
run: |
cp config/database.yml.ci config/database.yml
bundle exec rails db:create
bundle exec rails db:schema:load
- name: Run rspec
env:
RAILS_ENV: test
REDIS_HOST: redis
REDIS_PORT: ${{ job.services.redis.ports[6379] }}
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
run: bundle exec rspec --tag ~type:system
I am able to install ruby, node, the images, Postgres as a service, etc, and run Rubocop and Brakeman. But when I try to set up the DB before running Rspec it says it cannot connect to the DB.
As far as I've been able to ascertain, the host is localhost when running the VM configuration as opposed to a container configuration.
This is the database.yml.ci that the "Setup Database" step copies to the database.yml to be used by Rails.
test:
adapter: postgresql
encoding: unicode
database: db_test
pool: 5
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
host: <%= ENV['POSTGRES_HOST'] %>
I expected Postgres to be correctly set up and bundle exec rails db:create to create the database. However, it throws the following error:
rails aborted!
PG::ConnectionBad: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
I've tried all sorts of different configurations, but unfortunately, Actions is sort of knew and there doesn't seem to be a lot of material available online.
Any ideas on how to fix this?
===========================
EDIT:
So I was able to sort this out through trial and error. I ended up using a docker image with a ruby and node container. This is the working configuration:
on:
push:
branches:
- master
pull_request:
branches:
- master
- development
- release
jobs:
build:
runs-on: ubuntu-latest
container:
image: timbru31/ruby-node:latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ci_db_test
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
chrome:
image: selenium/standalone-chrome:latest
ports:
- 4444:4444
steps:
- uses: actions/checkout#v1
- name: Setup app dependencies
run: |
gem install bundler -v 1.17.3 --no-document
bundle install --jobs 4 --retry 3
npm install
npm install -g yarn
- name: Run rubocop
run: bundle exec rubocop
- name: Run brakeman
run: bundle exec brakeman
- name: Setup database
env:
RAILS_ENV: test
POSTGRES_HOST: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ci_db_test
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
run: |
cp config/database.yml.ci config/database.yml
bundle exec rails db:create
bundle exec rails db:schema:load
- name: Run rspec
env:
RAILS_ENV: test
POSTGRES_HOST: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ci_db_test
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}
SELENIUM_URL: 'http://chrome:4444/wd/hub'
run: bundle exec rspec
And the CI DB configuration database.yml.ci
default: &default
adapter: postgresql
encoding: unicode
username: <%= ENV['POSTGRES_USER'] %>
password: <%= ENV['POSTGRES_PASSWORD'] %>
host: <%= ENV['POSTGRES_HOST'] %>
pool: 5
database: <%= ENV['POSTGRES_DB'] %>
test:
<<: *default
I have a slightly different setup but this was the most relevant question when I encountered the same error so wanted to post here in case it can help. The two things that were critical for me were:
1) Set the DB_HOST=localhost
2) Set the --network="host" argument when you start the docker container with your rails app
name: Master Build
on: [push]
env:
registry: my_registry_name
# Not sure these are actually being passed down to rails, set them as the default in database.yml
DB_HOST: localhost
DB_USERNAME: postgres
DB_PASSWORD: postgres
jobs:
my_image_test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Check out repository
uses: actions/checkout#v2
- name: Build my_image docker image
uses: whoan/docker-build-with-cache-action#v5
with:
username: "${{secrets.aws_ecr_access_key_id}}"
password: "${{secrets.aws_ecr_secret_access_key}}"
registry: "${{env.registry}}"
image_name: my_image
context: my_image
- name: Lint rubocop
working-directory: ./my_image
run: docker run $registry/my_image bundle exec rubocop
- name: Run rails tests
working-directory: ./my_image
run: docker run --network="host" $registry/my_image bash -c "RAILS_ENV=test rails db:create && RAILS_ENV=test rails db:migrate && rails test"
Your problem appears to be that Postgres is not exposed on port 5432. Try to replace the port number with ${{ job.services.postgres.ports[5432] }}.
There are examples here: https://github.com/actions/example-services/blob/master/.github/workflows/postgres-service.yml
I had this challenge when trying to set up GitHub actions for a Rails Application.
Here's what worked for me:
name: Ruby
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
ruby-version:
- '2.7.2'
node-version:
- '12.22'
database-name:
- my-app
database-password:
- postgres
database-user:
- postgres
database-host:
- 127.0.0.1
database-port:
- 5432
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: ${{ matrix.database-name }}
POSTGRES_USER: ${{ matrix.database-user }}
POSTGRES_PASSWORD: ${{ matrix.database-password }}
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options:
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Check out Git Repository
uses: actions/checkout#v2
- name: Set up Ruby, Bundler and Rails
uses: ruby/setup-ruby#v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
- name: Set up Node
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: Install packages
run: |
yarn install --check-files
- name: Setup test database
env:
RAILS_ENV: test
DATABASE_NAME_TEST: ${{ matrix.database-name }}
DATABASE_USER: ${{ matrix.database-user }}
DATABASE_PASSWORD: ${{ matrix.database-password }}
DATABASE_HOST: ${{ matrix.database-host }}
DATABASE_PORT: ${{ matrix.database-port }}
POSTGRES_DB: ${{ matrix.database-name }}
run: |
bundle exec rails db:migrate
bundle exec rails db:seed
Note:
Replace my-app with the name of your app.
You can leave the database-password and database-user as postgres
That's all.
I hope this helps

CircleCI Swift with Postgres connection issues

I am working with my repo to build a test app for swift with circleCI and postgres but when it comes to testing I can't seem to grasp how to connect the two images in the testing phase.
I am running
circleci local execute --job build
Which should build both the swift and postgres images. I give them both the same env variables I give in the application. However I get this error when trying to run it. In my experience when trying to set up the two docker containers with compose this error was showing up when my api could not connect to the db container over the network.
Test Case 'AppTests.RemoveUserTest' started at 2019-04-09 19:46:15.380
Fatal error: 'try!' expression unexpectedly raised an error: NIO.ChannelError.connectFailed(NIO.NIOConnectionError(host: "db", port: 5432, dnsAError: Optional(NIO.SocketAddressError.unknown(host: "db", port: 5432)), dnsAAAAError: Optional(NIO.SocketAddressError.unknown(host: "db", port: 5432)), connectionErrors: [])): file /home/buildnode/jenkins/workspace/oss-swift-4.2-package-linux-ubuntu-16_04/swift/stdlib/public/core/ErrorType.swift, line 184
I know it says it failed because of a try statement but that try statement is failing because it's requesting actions from Postgres which is not there. Any ideas?
My current config.yml for circleci
version: 2
jobs:
build:
docker:
- image: swift:4.2
environment:
POSTGRES_USER: test
POSTGRES_PASSWORD: test
POSTGRES_DB: test
DB_HOSTNAME: db
PORT: 5432
- image: postgres:11.2-alpine
environment:
POSTGRES_USER: test
POSTGRES_PASSWORD: test
POSTGRES_DB: test
steps:
- checkout
- run: apt-get update -qq
- run: apt-get install -yq libssl-dev pkg-config wget
- run: apt-get install -y postgresql-client || true
- run:
name: install dockerize
command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
environment:
DOCKERIZE_VERSION: v0.3.0
- run:
name: Wait for db
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Compile code
command: swift build
- run:
name: Run unit tests
command: swift test
release:
docker:
- image: swift:4.2
steps:
- checkout
- run:
name: Compile code with optimizations
command: swift build -c release
push-to-docker-hub:
docker:
- image: docker:latest
steps:
- checkout
- setup_remote_docker
- run:
name: Install dependencies
command: |
apk add --update --no-cache curl jq python py-pip
- run:
name: Build Docker Image
command: |
docker build -t api .
docker tag api <>/repo:latest
docker tag api <>/repo:$CIRCLE_SHA1
docker login -u $DOCKER_USER -p $DOCKER_PASS
docker push <>/repo:latest
docker push <>/repo:$CIRCLE_SHA1
# - persist_to_workspace:
# root: ./
# paths:
# - k8s-*.yml
workflows:
version: 2
tests:
jobs:
- build
- push-to-docker-hub:
requires:
- build
context: dockerhub
filters:
branches:
only: master
#- linux-release
You're setting the hostname for the database to db, but not defining that anywhere. You need to name your Docker container to match the DB_HOSTNAME environment variable like so https://github.com/vapor/postgresql/blob/master/circle.yml#L8