To automate the configuration (docker run arguments) used to launch a docker container, I am writing a docker-compose.yml file.
My container should have access to the GPU, and so I currently use docker run --gpus=all parameter. This is described in the Expose GPUs for use docs:
Include the --gpus flag when you start a container to access GPU
resources. Specify how many GPUs to use. For example:
$ docker run -it --rm --gpus all ubuntu nvidia-smi
Unfortunately, Enabling GPU access with Compose doesn't describe this use case exactly. This guide uses the deploy yaml element, but in the context of reserving machines with GPUs. In fact, another documentation says that it will be ignored by docker-compose:
This only takes effect when deploying to a swarm with docker stack deploy, and is ignored by docker-compose up and docker-compose run.
After trying it and solving a myriad of problems along the way, I have realized that it is simply the documentation that is out of date.
Adding the following yaml block to my docker-compose.yml resulted in nvidia-smi being available to use.
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Related
I am trying to run jupyter Notebook inside of a docker.
My docker-compose is made of multiple in-house services and modules - in python - that need to be accessed in order to run different experimentations.
Should I just add a new service docker that is using the same network as the other services?
Will it be enough to use modules that are specified in the other services?
I'm supposing you want to access Jupyter from a Docker based image,
if so, you can use the base image from
https://hub.docker.com/r/jupyter/minimal-notebook/tags?page=1&name=3.8
with port forwarding to your localhost
For example:
docker run -it -p 8888:8888 jupyter/minimal-notebook:python-3.8.8
or run it with docker-compose
#docker-compose.yaml
version: '3.8'
services:
fjupyter:
image: jupyter/minimal-notebook:python-3.8.8
ports:
- 8888:8888
Using this base image, you can add all desired packages from bash but that wouldn't be the best approach since containerization dedicates each container for a specific service,
so it's better to use a dedicated image (hence a container) for each service.
Versions of Docker tooling:
Docker Compose version v2.6.0
Docker Engine Version 20.10.17 (client)
Docker Engine Version 20.10.17 (server)
I've got a compose file that starts a swarm with networks and secrets, so I'm using Docker Compose File Version 3.9.
One of my services is a GPU resource, so I added this based on current docs:
version: "3.9"
services:
my-app:
image: my-app:latest
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
count: all
. . .
My IDE is complaining with "expected scalar value" indicating that it doesn't recognize the key or value of capabilities and count and when I run the command:
docker stack deploy --compose-file docker-compose.yml my-stack
Docker throws an error:
services.prosit-app.deploy.resources.reservations Additional property devices is not allowed
Based on the versions of my Docker tools and the schema I'm using, this should all work (I think). What am missing?
I've tried various file versions and I can get the IDE warning to go away but then Docker can't support the file version. I've tried with and without quotes ["gpu"], different options, etc, to no avail.
This question was asked already (my bad) and the answer was missed by me and others:
Docker Compose returns error about property devices when trying to enable GPU
Reference:
https://docs.docker.com/compose/compose-file/compose-file-v3/#devices
I am starting to use kubernetes/Minikube to deploy my application which is currently running on docker containers.
Docker version:19.03.7
Minikube version: v1.25.2
From what I read I gather that first of all I need to build my frontend/backend images inside minikube.
The image is available on the server and I can see it using:
$ docker image ls
The first step, as far as I understand, is to use the "docker build" command:
$docke build -t my-image .
However, the dot at the end, so I understand, means it is looking for a Dockerfile in the curretn directoy, and indeed I get an error:
unable to evaluate symlinks in Dockerfile path: lstat
/home/dep/k8s-config/Dockerfile: no such file or directory
So, where do I get this dockerfile for the "docker build" to succeed?
Thanks
My missunderstanding...
I have the Dockefile now, so I should put it anywhere and use docker build from there.
I am running a local Kubernetes cluster through Docker Desktop on Windows. I'm attempting to modify my kube-apiserver config, and all of the information I've found has said to modify /etc/kubernetes/manifests/kube-apiserver.yaml on the master. I haven't been able to find this file, and am not sure what the proper way is to do this. Is there a different process because the cluster is through Docker Desktop?
Is there a different process because the cluster is through Docker Desktop?
You can get access to the kubeapi-server.yaml with a Kubernetes that is running on Docker Desktop but in a "hacky" way. I've included the explanation below.
For setups that require such reconfigurations, I encourage you to use different solution like for example minikube.
Minikube has a feature that allows you to pass the additional options for the Kubernetes components. You can read more about --extra-config ExtraOption by following this documentation:
Minikube.sigs.k8s.io: Docs: Commands: Start
As for the reconfiguration of kube-apiserver.yaml with Docker Desktop
You need to run following command:
docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh
Above command will allow you to run:
vi /etc/kubernetes/manifests/kube-apiserver.yaml
This lets you edit the API server configuration. The Pod running kubeapi-server will be restarted with new parameters.
You can check below StackOverflow answers for more reference:
Stackoverflow.com: Answer: Where are the Docker Desktop for Windows kubelet logs located?
Stackoverflow.com: Answer: How to change the default nodeport range on Mac (docker-desktop)?
I've used this answer without $ screen command and I was able to reconfigure kubeapi-server on Docker Desktop in Windows
docker-compose down does not have a [SERVICE...] argument.
Per docker-compose down --help:
Usage: down [options]
I would like to be able to bring just one of my many containers down. I know that I can down a single container using docker down instead of docker-compose down, but I'm wondering why doesn't the docker-compose down command take an optional [SERVICE...] argument?
All of the following docker-compose commands do take an optional [SERVICE...] argument:
docker-compose build
docker-compose create
docker-compose kill
docker-compose logs
docker-compose pause
docker-compose restart
docker-compose rm
docker-compose start
docker-compose stop
docker-compose unpause
docker-compose up
My docker-compose --version is 1.9.0
The following command is the equivalent of docker down for a single service:
docker-compose rm -s -v my_service
Usage: rm [options] [SERVICE...]
Options:
-s, --stop Stop the containers, if required, before removing
-v Remove any anonymous volumes attached to containers
There is no docker down. The corresponding docker command is closer to: docker stop; docker rm
From docker-compose down:
Stops containers and removes containers, networks, volumes, and images created by up.
By default, the only things removed are:
Containers for services defined in the Compose file
Networks defined in the networks section of the Compose file
The default network, if one is used
If docker-compose down removes networks also, then by removing one container, it should also automatically disconnect the other containers from these networks, which might be undesirable or confusing.
It could change so that by default docker down does not remove the networks, but that change might cause backwards-compatibility issues with the command.
This is just an assumption.