In Concourse CI, by default, the underlying container for a job's task is instantiated and run with user root.
If the container used for my task needs to be executed with a different user (e.g. postgres), how can I do that in Concourse?
Concourse tasks provide a user parameter to explicitly set the user to run its container as.
See http://concourse-ci.org/running-tasks.html#task-run-user .
Here is a sample Concourse pipeline to demonstrate the use of that parameter:
---
jobs:
- name: check-container-user
plan:
- do:
- task: container-user-postgres
config:
platform: linux
image_resource:
type: docker-image
source:
repository: postgres
tag: "latest"
run:
user: postgres
path: sh
args:
- -exc
- |
whoami
echo "Container running with postgres user"
Related
I am building a CI workflow using GitHub Actions.
Goal is to build and test a .NET C# application using a MSSQL database.
The database can successfully start and the database is created. In the logs of the DB container I can see the DB was created and is running.
In the testing step all tests fail with this error:
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Resource temporarily unavailable. Aborting test execution.
The database name is correctly given to the program. I have tested this by printing the connection string to the console. Also it actually connects to the DB (because when the database server name is incorrent I get a error that reflects that).
The CI workflow:
name: .NET Backend Build and run Unit Tests
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build_and_test:
runs-on: ubuntu-latest
services:
sqlserver:
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- 1433:1433
env:
ACCEPT_EULA: "Y"
SA_PASSWORD: "redacted"
MSSQL_PID: "Express"
MSSQL_COLLATION: "SQL_Latin1_General_CP1_CI_AS"
steps:
- name: get Container ID
run: echo "DATABASE_SERVER=$(docker ps --all --filter status=running --format "{{.ID}}")" >> $GITHUB_ENV
- name: create database
run: docker exec $DATABASE_SERVER /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'redacted' -Q 'CREATE DATABASE dbname'
- uses: actions/checkout#v3
- name: Setup .NET
uses: actions/setup-dotnet#v3
with:
dotnet-version: 6.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore
- name: Test
env:
DATABASE_PORT: 1433
DATABASE_NAME: dbname
DATABASE_USER: sa
DATABASE_PASSWORD: redacted
run: dotnet test --no-build --verbosity normal
I found out the Docker in GitHub Actions does not support DNS and behaves fundamentally different than regular Docker on Linux. All network communication must be made over the host network. So I had to specify localhost as the database server.
This would be the correct workflow file:
name: .NET Backend Build and run Unit Tests
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
env:
CI: true
jobs:
build_and_test:
runs-on: ubuntu-latest
services:
sqlserver:
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- 1433:1433
env:
ACCEPT_EULA: "Y"
MSSQL_SA_PASSWORD: "redacted"
MSSQL_PID: "Express"
MSSQL_COLLATION: "SQL_Latin1_General_CP1_CI_AS"
steps:
- name: get Container ID
run: echo "DATABASE_ID=$(docker ps --all --filter status=running --format "{{.ID}}")" >> $GITHUB_ENV
- name: create database
run: docker exec $DATABASE_ID /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'redacted' -Q 'CREATE DATABASE dbname'
- uses: actions/checkout#v3
- name: Setup .NET
uses: actions/setup-dotnet#v3
with:
dotnet-version: 6.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore
- name: Test
env:
DATABASE_SERVER: localhost
DATABASE_PORT: 1433
DATABASE_NAME: dbname
DATABASE_USER: sa
DATABASE_PASSWORD: redacted
run: dotnet test --no-build --verbosity normal
I'm trying to run a container that runs a program until it finishes, as a step in Azure DevOps pipeline Job.
From documentation it looks that what's needed is a service container.
My pipeline yaml is:
trigger:
- main
resources:
containers:
- container: mycontainer
image: mycontainer:latest
endpoint: myregistry
pool:
vmImage: ubuntu-latest
services:
syncice: mycontainer
steps:
- script: |
ls
printenv
When the container is docker run locally the program shows output, but from DevOps Job no output is showing.
How to start the container and see output in Job?
Wanted to know if there is any flag/option for concourse tasks inside a single job so that all tasks gets executed regardless of any task failing.
Thanks!
Totally. By default, tasks run sequentially. If you want them to run independently of the sequence place them in the in_parallel key, like in the following pipeline:
jobs:
- name: parallel-tasks
plan:
- in_parallel:
- task: failing-task
config:
platform: linux
image_resource:
type: docker-image
source:
repository: alpine
run:
path: /bin/sh
args: [ "-c", "exit 1"]
- task: passing-task
config:
platform: linux
image_resource:
type: docker-image
source:
repository: alpine
run:
path: /bin/sh
args: [ "-c", "exit 0"]
Running it will produce the following output:
in_parallel works with tasks as well as resources (e.g. running get in parallel)
I am playing around with Azure DevOps container jobs and service containers. My use case is as follows, I (unfortunately) have to do everything on Private Hosted Build agents.
I am running my job as a container job in Container A.
I have specific software installed (Fortify), which uses commandline, on Container B
Basically I want one of the steps running on container A to be run in Container B (to do the fortify scan, using the code from the workspace). Of course I could do it in a separate job, but I'd prefer to do it in the same job.
Any ideas if this is possible at the moment?
Thanks
Cool, I just read that this feature will be available in the sprint 163 release!
https://learn.microsoft.com/en-us/azure/devops/release-notes/2020/sprint-163-update
resources:
containers:
- container: python
image: python:3.8
- container: node
image: node:13.2
jobs:
- job: example
container: python
steps:
- script: echo Running in the job container
- script: echo Running on the host
target: host
- script: echo Running in another container, in restricted commands mode
target:
container: node
commands: restricted
You can use the Step target to choose which container or host the step will running at.
For example:
resources:
containers:
- container: pycontainer
image: python:3.8
steps:
- task: SampleTask#1
target: host
- task: AnotherTask#1
target: pycontainer
I'm trying to set up a CI/CD pipeline in GitHub Actions for my Elixir project.
I can fetch dependencies, compile them, check formatting, credo... But when the tests starts, I'm not able to reach the PostgreSQL service declared on the YAML.
How can I link both containers? (Elixir and PostgreSQL)
According to the logs shown on GitHub Actions, both containers are on the same Docker network, so they should be reachable from each other using their network aliases. However, when I try to connect to the postgres one, it says NXDOMAIN. Also the ping doesn't work, as expected.
The content of my workflow:
name: Elixir CI
on: push
jobs:
build:
runs-on: ubuntu-18.04
container:
image: elixir:1.9.1
services:
postgres:
image: postgres
ports:
- 5432:5432
env:
POSTGRES_USER: my_app
POSTGRES_PASSWORD: my_app
POSTGRES_DB: my_app_test
steps:
- uses: actions/checkout#v1
- name: Install Dependencies
env:
MIX_ENV: test
run: |
cp config/test.secret.ci.exs config/test.secret.exs
mix local.rebar --force
mix local.hex --force
apt-get update -qqq && apt-get install make gcc -y -qqq
mix deps.get
- name: Compile
env:
MIX_ENV: test
run: mix compile --warnings-as-errors
- name: Run formatter
env:
MIX_ENV: test
run: mix format --check-formatted
- name: Run Credo
env:
MIX_ENV: test
run: mix credo
- name: Run Tests
env:
MIX_ENV: test
run: mix test
Also, on Elixir I have set up the test task to connect to postgres:5432, but it says the host does not exist.
According to some tutorials and examples I found on the Internet, this configurations looks like valid, but nothing I could do made it work.
You need to pass the name of the service ("postgres") as POSTGRES_HOST to the application and set the port POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }} (spaces matter.)
Github CI dynamically routes port and host to it.
I wrote a blog post on the subject a couple of days ago.