Does docker-compose version 1.29.1 support the --gpus parameter? If not, are there other parameters that support setting up the use of gpu in docker-compose 1.29.1 ?
--gpus argument works with docker command. For docker-compose GPU configuration is done within docker-compose.yml:
Using the runtime option (legacy v2.3 format)
version: "2.3"
services:
test:
image: nvidia/cuda:10.2-base
command: nvidia-smi
runtime: nvidia # <- this option
Using the device structure (since docker-compose v1.28.0)
version: "3.8"
services:
test:
image: nvidia/cuda:10.2-base
command: nvidia-smi
deploy:
resources:
reservations:
devices:
- driver: nvidia
# that's the closest analogue to --gpus; provide
# an integer amount of devices or 'all'
count: 1
# Devices are reserved using a list of capabilities, making
# capabilities the only required field. A device MUST
# satisfy all the requested capabilities for a successful
# reservation.
capabilities: [gpu]
The latter might seem a bit complicated but there is a guide the explains both (Enabling GPU access with Compose
) and some extra information can be obtained from The Compose Specification.
Related
Versions of Docker tooling:
Docker Compose version v2.6.0
Docker Engine Version 20.10.17 (client)
Docker Engine Version 20.10.17 (server)
I've got a compose file that starts a swarm with networks and secrets, so I'm using Docker Compose File Version 3.9.
One of my services is a GPU resource, so I added this based on current docs:
version: "3.9"
services:
my-app:
image: my-app:latest
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
count: all
. . .
My IDE is complaining with "expected scalar value" indicating that it doesn't recognize the key or value of capabilities and count and when I run the command:
docker stack deploy --compose-file docker-compose.yml my-stack
Docker throws an error:
services.prosit-app.deploy.resources.reservations Additional property devices is not allowed
Based on the versions of my Docker tools and the schema I'm using, this should all work (I think). What am missing?
I've tried various file versions and I can get the IDE warning to go away but then Docker can't support the file version. I've tried with and without quotes ["gpu"], different options, etc, to no avail.
This question was asked already (my bad) and the answer was missed by me and others:
Docker Compose returns error about property devices when trying to enable GPU
Reference:
https://docs.docker.com/compose/compose-file/compose-file-v3/#devices
To automate the configuration (docker run arguments) used to launch a docker container, I am writing a docker-compose.yml file.
My container should have access to the GPU, and so I currently use docker run --gpus=all parameter. This is described in the Expose GPUs for use docs:
Include the --gpus flag when you start a container to access GPU
resources. Specify how many GPUs to use. For example:
$ docker run -it --rm --gpus all ubuntu nvidia-smi
Unfortunately, Enabling GPU access with Compose doesn't describe this use case exactly. This guide uses the deploy yaml element, but in the context of reserving machines with GPUs. In fact, another documentation says that it will be ignored by docker-compose:
This only takes effect when deploying to a swarm with docker stack deploy, and is ignored by docker-compose up and docker-compose run.
After trying it and solving a myriad of problems along the way, I have realized that it is simply the documentation that is out of date.
Adding the following yaml block to my docker-compose.yml resulted in nvidia-smi being available to use.
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Since I'm developing on an M1 mac, my Docker builds will build ARM. I want to build x86_64 images which I know I can build with the --platform flag but I want to do that with my Skaffold configuration.
Thanks to #p10l putting me on the right track, I was able to figure it out. Skaffold can't set the platform using the Docker Engine API, so we need to pass --platform=linux/x86_64 to the command line by setting useDockerCLI to true and adding cliFlags to our Docker configuration.
apiVersion: skaffold/v2beta26
kind: Config
build:
# ... other unrelated config ...
artifacts:
- image: my-image
context: ./
docker:
cliFlags:
- --platform=linux/x86_64
local:
useDockerCLI: true # the only way to set platform is with cliFlags so we need to enable the Docker CLI
Whoever is coming here from google and confront this problem, skaffold almost finished developing multi-arch support but as the version of v1.39.1, you can use
skaffold run --platform linux/amd64
or simply to configure it in skaffold.yaml file
Skaffold has buildCommand field to which you can pass a custom script.
So, for example
...
build:
artifacts:
- image: "foo"
context: .
custom:
buildCommand: ./build.sh
...
build.sh
docker buildx build \
--platform linux/amd64
... # any other flags
Disclaimer: Everything below is a speculation. I'm currently unable to test this, but I'm sure someone will correct me if I'm wrong.
There is also build.artifacts.docker field (currently in beta). It may be possible to use this field to pass arguments to Docker build.
...
build:
artifacts:
- image: "foo"
context: .
docker:
dockerfile: <Dockerfile relative to workspace>
target: <Dockerfile target name to build>
buildArgs:
platform: linux/amd64
local:
useDockerCLI: true #this is needed to use docker build CLI rather than Docker Engine API
It may also be required to set buildx as a default builder for Docker. This can be achieved with
docker buildx install
version: '3.8'
services:
foo:
...
networks:
- $FOO_NETWORK
networks:
foo_network:
I am unable to use $FOO_NETWORK under networks, i.e. it allows only to enter a value and not an ENV variable. How do I customize the network name to be taken from the environment variable instead
Environment variables are for values, you want to use it for a key. As far as i know this isn't supported yet and I'm not sure if it'll ever be.
One way you can customise this is to use multiple docker-compose files. Create three files:
one.yml:
version: "3.0"
services:
test:
image: nginx
two.yml:
version: "3.0"
services:
test:
networks:
foo: {}
networks:
foo: {}
three.yml:
version: "3.0"
services:
test:
networks:
bar: {}
networks:
bar: {}
Now you if you run it like this:
docker-compose -f one.yml -f two.yml up
or like this:
docker-compose -f one.yml -f three.yml up
You'll see that the files are merged:
Creating network "network_foo" with the default driver
Recreating network_test_1 ... done
...
Creating network "network_bar" with the default driver
Recreating network_test_1 ... done
You can even spin all three at once:
docker-compose -f one.yml -f two.yml -f three.yml up
Creating network "network_foo" with the default driver
Creating network "network_bar" with the default driver
Creating network_test_1 ... done
Attaching to network_test_1
test_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
Check out documentation for more: https://docs.docker.com/compose/extends/
Also there is another way, which actually involves using variables to select a network. The way is to use existing networks. You'll need an .env file for this:
network=my_network
and in compose file you do like this:
version: "3.8"
services:
test:
networks:
mynet: {}
networks:
mynet:
external: true
name: $network
As you see there is an option to provide name when using an external network. The network with the name must exist when you start your containers or you'll get an error. You can use a separate file to create networks on a node or just create using CLI. Note that the compose version changed, the feature isn't supported in "3.0".
docker-compose --version
docker-compose version 1.11.1, build 7c5d5e4
I have secret 'my_secret_data' added to my swarm cluster:
The start of my compose file looks like:
version: "3.1"
secrets:
my_secret_data:
external: true
services:
master:
image: jenkins-master
secrets:
- my_secret_data
ports:
- "8080:8080"
- "50000:50000"
'docker stack deploy' continually gives the error:
secrets Additional property secrets is not allowed
I have followed how do you manage secret values with docker-compose v3.1? to the letter as far as I can tell and have the correct versions installed but keep getting the above error. Any help greatly appreciated.
Change compose file version to latest version.
In short, version '3' is not resolved to the latest '3.x' version. Find what the latest version is here https://docs.docker.com/compose/compose-file/#compose-and-docker-compatibility-matrix
The "Additional property secrets is not allowed" error can be caused either by:
running Docker Engine < 1.13.1, or
using a compose file version number < '3.1' in a docker-compose file such as docker-compose.yml or docker-cloud.yml
If you are experiencing this problem confirm that both are correct.
This also applies to other Docker interfaces and tools.
For examples, in Portainer, yml with secrets lines pasted into the Create Stack dialog should begin with the line version: '3.1' or you will encounter the same error -- even with an up-to-date Docker Engine 1.13.1+.
In my case, Service: had an extra tab prior. Moment I removed tab prior to it, it worked.