I'm trying to run a container that runs a program until it finishes, as a step in Azure DevOps pipeline Job.
From documentation it looks that what's needed is a service container.
My pipeline yaml is:
trigger:
- main
resources:
containers:
- container: mycontainer
image: mycontainer:latest
endpoint: myregistry
pool:
vmImage: ubuntu-latest
services:
syncice: mycontainer
steps:
- script: |
ls
printenv
When the container is docker run locally the program shows output, but from DevOps Job no output is showing.
How to start the container and see output in Job?
Related
I'm trying to use service containers within Azure DevOps pipelines
The agent is an ubuntu host
I would like to have the agent run a powershell container and a playwright container
The doc for this is not very verbose
So far I have this in my main 'azure-pipelines.yml'
trigger: none
pr: none
resources:
containers:
- container: playwright
image: mcr.microsoft.com/playwright:v1.29.0-focal
- container: pwsh
image: mcr.microsoft.com/powershell
pool:
vmImage: "ubuntu-latest"
services:
playwright: playwright
pwsh: pwsh
stages:
- stage: dev
displayName: dev
jobs:
- template: templates/test.yml
And this in my 'template/test.yml' file
- job: run_tests
displayName: Test
pool:
vmImage: ubuntu-latest
steps:
- powershell: |
Write-Host "This is powershell"
target:
container: pwsh
- script: yarn test:integration:ci
displayName: "Run tests"
env:
environment: dev
CI: true
target:
container: playwright
Azure pipelines does not like this. It is failing with:
/.azure/azure-pipelines.yml (Line: 18, Col: 1): Unexpected value 'stages'
when I try to run the pipeline. I thought stages: was the first part of a pipeline? (but I am very new to Azure pipelines so my understanding might be way off)
Could anyone help to clarify why/where I am screwing up at all please?
Thanks
Make the following changes to your yaml files.
azure-pipelines.yml
trigger: none
pr: none
pool:
vmImage: ubuntu-latest
resources:
containers:
- container: playwright
image: mcr.microsoft.com/playwright:v1.29.0-focal
- container: pwsh
image: mcr.microsoft.com/powershell
stages:
- stage: dev
displayName: dev
jobs:
- template: templates/test.yml
template.yml
jobs:
- job: run_tests
displayName: Test
pool:
vmImage: ubuntu-latest
services:
playwright: playwright
pwsh: pwsh
steps:
- powershell: |
Write-Host "This is powershell"
target:
container: pwsh
- script: yarn test:integration:ci
displayName: "Run tests"
env:
environment: dev
CI: true
target:
container: playwright
Reason why you were getting the error (/.azure/azure-pipelines.yml (Line: 18, Col: 1): Unexpected value 'stages') is because of the property services. According to the example in service containers documentation, the property services is defined in the root level of the yaml because the example did not use any stage or jobs.
Since you are using stages and jobs in your yaml pipeline, the services property should be nested within your job.
Hence, I have moved the services to the template.yml file. You can check which property is allowed under a stage or job using this YAML schema documentation
Reference: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/jobs-job-container?view=azure-pipelines
We are currently using CircleCI to run our automated tests and would like to migrate to Azure DevOps to run those tests on an Azure Pipeline. Our applications are fully dockerized and I am having trouble executing the tests in the container on Azure Pipeline.
My goal is simply to build the image, push it to our Docker Hub repo and then pull it to execute PHPUnit. The first part is OK, I managed to push the image.
Then I created a job to execute a simple script, and I would like for it to run inside the container. My pipeline conf file will follow. The step that fails currently is the container initialization of the second job. It fails with the error :
/usr/bin/docker pull [redacted]:azure-master
Error response from daemon: pull access denied for [redacted], repository does not exist or may require 'docker login': denied: requested access to the resource is denied
trigger:
- master
resources:
- repo: self
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build_and_push
displayName: Build and push image
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Docker#2
displayName: Build and push image
inputs:
containerRegistry: 'Docker Hub'
repository: '[redacted]'
command: 'buildAndPush'
Dockerfile: '**/Dockerfile'
tags: 'azure-$(Build.SourceBranchName)'
- task: Docker#2
displayName: Login to docker repo
inputs:
containerRegistry: 'Docker Hub'
command: 'login'
- job: Install_composer_and_run_tests
dependsOn: ['Build_and_push']
pool:
vmImage: 'ubuntu-latest'
container: [redacted]:azure-$(Build.SourceBranchName)
steps:
- task: Docker#2
displayName: Login to docker repo
inputs:
containerRegistry: 'Docker Hub'
command: 'login'
- script: composer install -n --prefer-dist
- script: php vendor/bin/phpunit tests/ --group me
I don't really understand how or where I should login because I use the container param in the job, not a task to pull the image. Plus I have not problem pushing the image even though I did not explicitly login at that step. Last thing is that I have created a container registry in Azure DevOps (Docker Hub), with my credentials and it works correctly.
Thanks for your help :)
See Endpoints:
Containers can be hosted on registries other than Docker Hub. To host an image on Azure Container Registry or another private container registry, add a service connection to the private registry. Then you can reference it in a container spec:
container:
image: xxx/xxx:tag
endpoint: xxx
According to your error message, you may need to provide credentials for the Initialize Containers step. So we should use this format:
- job: Install_composer_and_run_tests
dependsOn: ['Build_and_push']
pool:
vmImage: 'ubuntu-latest'
container:
image: [redacted]:azure-$(Build.SourceBranchName)
endpoint: 'Docker Hub'
steps:
...
I am playing around with Azure DevOps container jobs and service containers. My use case is as follows, I (unfortunately) have to do everything on Private Hosted Build agents.
I am running my job as a container job in Container A.
I have specific software installed (Fortify), which uses commandline, on Container B
Basically I want one of the steps running on container A to be run in Container B (to do the fortify scan, using the code from the workspace). Of course I could do it in a separate job, but I'd prefer to do it in the same job.
Any ideas if this is possible at the moment?
Thanks
Cool, I just read that this feature will be available in the sprint 163 release!
https://learn.microsoft.com/en-us/azure/devops/release-notes/2020/sprint-163-update
resources:
containers:
- container: python
image: python:3.8
- container: node
image: node:13.2
jobs:
- job: example
container: python
steps:
- script: echo Running in the job container
- script: echo Running on the host
target: host
- script: echo Running in another container, in restricted commands mode
target:
container: node
commands: restricted
You can use the Step target to choose which container or host the step will running at.
For example:
resources:
containers:
- container: pycontainer
image: python:3.8
steps:
- task: SampleTask#1
target: host
- task: AnotherTask#1
target: pycontainer
I manage to use them fine as long as I don't need to pass custom arguments.
Lets say I want to use an official Docker image: somePublicImage:1.2.3; then the following works fine:
stages:
- stage: Build
jobs:
- job: BuildTestPack
displayName: 'Build, test & pack'
timeoutInMinutes: 5
cancelTimeoutInMinutes: 2
services:
someService:
image: somePublicImage:1.2.3
ports:
- 4223:4222
There's an option to configure the container with --foo bar
How do I define this in a Azure build pipeline?
I've tried:
command
options
arguments
entrypoint
Service containers must define a CMD or ENTRYPOINT. The pipeline will docker run the provided container without additional arguments.
Check the link below:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/service-containers?view=azure-devops&tabs=yaml
Seems like you need to create a "custom" resource container first. E.g
resources:
containers:
- container: myThing
image: somePublicImage:1.2.3
ports:
- 4223:4222
volumes:
- /docker_vol_config:/config
command: '--foo bar'
which then can be used as a service:
stages:
- stage: Build
jobs:
- job: BuildTestPack
displayName: 'Build, test & pack'
timeoutInMinutes: 5
cancelTimeoutInMinutes: 2
services:
myThing:myThing
In Concourse CI, by default, the underlying container for a job's task is instantiated and run with user root.
If the container used for my task needs to be executed with a different user (e.g. postgres), how can I do that in Concourse?
Concourse tasks provide a user parameter to explicitly set the user to run its container as.
See http://concourse-ci.org/running-tasks.html#task-run-user .
Here is a sample Concourse pipeline to demonstrate the use of that parameter:
---
jobs:
- name: check-container-user
plan:
- do:
- task: container-user-postgres
config:
platform: linux
image_resource:
type: docker-image
source:
repository: postgres
tag: "latest"
run:
user: postgres
path: sh
args:
- -exc
- |
whoami
echo "Container running with postgres user"