Is there any kubernetes api or kubectl command to delete older docker images that is lying on the device.
I know we can delete by using docker rm image but i want to do remotely through API.
Any alternative?
The kubelet removes unused images automatically when the docker disk fullness reaches a configurable threshold.
See https://kubernetes.io/docs/concepts/architecture/garbage-collection/#containers-images for details on how that works.
There is no API-lead way to forcefully delete an image.
If you really need to manually clean up an image from nodes, you could run a container that connects to the docker daemon and runs docker rmi <image> there, but it smells like an antipattern to me.
There is no kubernetes api or kubectl command to delete older docker images.
You should be using
docker rmi <image>
or
docker system prune
commands to remove unwanted images.
Related
I have a docker-compose.yml from which I started a couple of services. I add a new volume mapping to one of the services and then try to restart the container with
docker compose restart <service_name>
but the volume is still not mapped and not available from within the image.
What is the right way to add a volume to an image defined with docker compose?
Oki, so it turns out that restart is just a refresh of the existing image but changes nothing in the parameters with which it is started.
In order to have compose take into account volume mapping changes in the docker-compose.yml file one has ro run:
docker compose up --build <service_name>
There might be other solutions, but this is what I ended up doing.
I'm aware that if I change my Dockerfile or build directory, I'm supposed to run docker-compose build. This surely implies that docker-compose has some cache somewhere of its already-built images.
Where is it? How do I purge it?
I'd like to get back to a state where docker-compose up is forced to do the initial build steps, without me needing to remember to run docker-compose build.
I've run docker stop $(docker ps -aq) and docker X prune (for X in container, image, volume, network), but docker-compose up still refuses to run the build steps in my Dockerfile.
Or am I completely misunderstanding how docker-compose works?
you can pass on additional argument (--no-cache) to skip using cache during build process.
docker#default:~$ docker-compose build --help
Build or rebuild services.
Services are built once and then tagged as `project_service`,
e.g. `composetest_db`. If you change a service's `Dockerfile` or the
contents of its build directory, you can run `docker-compose build` to rebuild it.
Usage: build [options] [--build-arg key=val...] [SERVICE...]
Options:
--compress Compress the build context using gzip.
--force-rm Always remove intermediate containers.
--no-cache Do not use cache when building the image.
--pull Always attempt to pull a newer version of the image.
-m, --memory MEM Sets memory limit for the build container.
--build-arg key=val Set build-time variables for services.
docker#default:~$
docker-compose uses images, which you can see with docker images:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker_ubuntu latest a7dc4f9bbdfb 19 hours ago 158MB
ubuntu 16.04 0b1edfbffd27 3 weeks ago 113MB
hello-world latest f2a91732366c 6 months ago 264MB
The docker-compose images are prefixed with (usually) the name of the directory you're running docker-compose in. So, for me, the docker_ubuntu image.
docker image prune thinks that the images are in use, so it doesn't prune them.
To get rid of the docker-compose image, you need to delete it explicitly:
docker image rm docker_ubuntu
I'm new to docker. I'm still trying to wrap my head around all this.
I'm building a node application (REST api), using Postgresql to store my data.
I've spent a few days learning about docker, but I'm not sure whether I'm doing things the way I'm supposed to.
So here are my questions:
I'm using the official docker postgres 9.5 image as base to build my own (my Dockerfile only adds plpython on top of it, and installs a custom python module for use within plpython stored procedures). I created my container as suggedsted by the postgres image docs:
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
After I stop the container I cannot run it again using the above command, because the container already exists. So I start it using docker start instead of docker run. Is this the normal way to do things? I will generally use docker run the first time and docker start every other time?
Persistance: I created a database and populated it on the running container. I did this using pgadmin3 to connect. I can stop and start the container and the data is persisted, although I'm not sure why or how is this happening. I can see in the Dockerfile of the official postgres image that a volume is created (VOLUME /var/lib/postgresql/data), but I'm not sure that's the reason persistance is working. Could you please briefly explain (or point to an explanation) about how this all works?
Architecture: from what I read, it seems that the most appropriate architecture for this kind of app would be to run 3 separate containers. One for the database, one for persisting the database data, and one for the node app. Is this a good way to do it? How does using a data container improve things? AFAIK my current setup is working ok without one.
Is there anything else I should pay atention to?
Thanks
EDIT: adding to my confusion, I just ran a new container from the debian official image (no Dockerfile, just docker run -i -t -d --name debtest debian /bin/bash). With the container running in the background, I attached to it using docker attach debtest and the proceeded to apt-get install postgresql. Once installed I ran (still from within the container) psql and created a table in the default postgres database, and populated it with 1 record. Then I exited the shell and the container stopped automatically since the shell wasn't running anymore. I started the container againg using docker start debtest, then attached to it and finally run psql again. I found everything is persisted since the first run. Postgresql is installed, my table is there, and offcourse the record I inserted is there too. I'm really confused as to why do I need a VOLUME to persist data, since this quick test didn't use one and everything apears to work just fine. Am I missing something here?
Thanks again
1.
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword
-d postgres
After I stop the container I cannot run it again using the above
command, because the container already exists.
Correct. You named it (--name some-postgres) hence before starting a new one, the old one has to be deleted, e.g. docker rm -f some-postgres
So I start it using
docker start instead of docker run. Is this the normal way to do
things? I will generally use docker run the first time and docker
start every other time?
No, it is by no means normal for docker. Docker process containers are supposed normally to be ephemeral, that is easily thrown away and started anew.
Persistance: ... I can stop and start
the container and the data is persisted, although I'm not sure why or
how is this happening. ...
That's because you are reusing the same container. Remove the container and the data is gone.
Architecture: from what I read, it seems that the most appropriate
architecture for this kind of app would be to run 3 separate
containers. One for the database, one for persisting the database
data, and one for the node app. Is this a good way to do it? How does
using a data container improve things? AFAIK my current setup is
working ok without one.
Yes, this is the good way to go by having separate containers for separate concerns. This comes in handy in many cases, say when for example you need to upgrade the postgres base image without losing your data (that's in particular where the data container starts to play its role).
Is there anything else I should pay atention to?
When acquainted with the docker basics, you may take a look at Docker compose or similar tools that will help you to run multicontainer applications easier.
Short and simple:
What you get from the official postgres image is a ready-to-go postgres installation along with some gimmicks which can be configured through environment variables. With docker run you create a container. The container lifecycle commands are docker start/stop/restart/rm Yes, this is the Docker way of things.
Everything inside a volume is persisted. Every container can have an arbitrary number of volumes. Volumes are directories either defined inside the Dockerfile, the parent Dockerfile or via the command docker run ... -v /yourdirectoryA -v /yourdirectoryB .... Everything outside volumes is lost with docker rm. Everything including volumes is lost with docker rm -v
It's easier to show than to explain. See this readme with Docker commands on Github, read how I use the official PostgreSQL image for Jira and also add NGINX to the mix: Jira with Docker PostgreSQL. Also a data container is a cheap trick to being able to remove, rebuild and renew the container without having to move the persisted data.
Congratulations, you have managed to grasp the basics! Keep it on! Try docker-compose to better manage those nasty docker run ...-commands and being able to manage multi-containers and data-containers.
Note: You need a blocking thread in order to keep a container running! Either this command must be explicitly set inside the Dockerfile, see CMD, or given at the end of the docker run -d ... /usr/bin/myexamplecommand command. If your command is NON blocking, e.g. /bin/bash, then the container will always stop immediately after executing the command.
I'd like to make the deployment to production the easiest it can be, but struggling with the way how to do it.
If I will have docker for production, it will be nice to have docker image with my application deployables, but I'm not sure if it is good approach.
I have several concerns:
wouldn't the layer system bloat, when I will replace the file every time in new version of image?
Is it good idea to make DB scripts and migration tool part of this image?
The last concern is how to run it conveniently. I don't want to go there stop the tomcat container and start it again using volume from new application image(as the new app container name cannot be the same).
I have seen ways to do that, but I don't like them very much i.e. deploy to Tomcat docker image ,create Tomcat image with application already bundled or use host system volume. I like to have something like install "CD". I'd like to evaluate my idea with other approaches, speaking about the proper tool to run it is maybe for other question.
wouldn't the layer system bloat, when I will replace the file every time in new version of image?
No because you can clean up dangling images
docker rmi $(docker images --filter "dangling=true" -q --no-trunc)
Is it good idea to make DB scripts and migration tool part of this image?
Yes, if your startup script knows to detect if it needs to apply them.
I don't like them very much i.e. deploy to Tomcat docker image ,create Tomcat image with application already bundled or use host system volume.
If your data volume container is separate from the app, that shouldn't be an issue.
From the discussion, the OP adds:
using this docker create --name <container_name> <image_name> with different image name can retain the container name and I can run Tomcat container with the same volumes-from?
docker run -it --rm -p 8888:8080 --volumes-from <container_name> <image_name>
That is the idea, but it won't work if there is already a create data container with that name.
If there is no persistent data in it, one can docker rm that data container, and recreate it with the same name.
If there are persistent data, then it is best to copy the new updated data through an intermediate (docker run) container which would mount temporarily the data container.
My question aims at verifying and maybe rectifying my idea of the reliability of Docker containers. I read both, the Docker documentation and several articles on VOLUME in the Dockerfile and --v as an argument when running a container as means to persist data outside a Docker container. Be it in a data container or on the host system. As would like to keep the complexity of my setup simple, I would prefer not to copy/save/store data round and about but keep it in the Docker container itself.
There are several cases through which I discovered the behaviour of Docker containers. I'd like to know if I missed a scenario where a container can be 100% lost unpurposely, i.e. NOT doing $ docker rm -f mycontainer
docker commands to pause, stop and kill a container
-> restartable by $ docker restart mycontainer or $ docker run mycontainer
Host system reboot
-> docker container exits with 0 or 255
Host system unexpected power off
-> What happens?
Application exception
-> docker container exits with -1
Updating or restarting docker (as pointed out by Greg)
-> expected behavior: like on system reboot (?)
In all those cases, the docker container is still existent in the end. So is there any other scenario that can cause a docker container to be lost like with $ docker rm -f mycontainer?
The background is, that I read a lot about mounted volumes and external datastorage on the host system for Postgres but I'd like to avoid storing data outside my containers on the host system if possible. On the other hand, I don't want to wake up and have all data lost. (I do perform regular SQL-dumps, but I don't want to do this every 5 minutes). If a docker container itself is not reliable for persistant data, I don't see why I should create a second container to hold the data for a first one and increase the complexity of my system by adding a new container but not gaining anything in terms of reliability.
Edit: There are two points in the Docker userguide on Volumes which do not explicitly explain which behaviour to expect and therefore making me question if these concepts provide extra reliability:
Changes to a data volume will not be included when you update an
image
-> Does that mean that they get lost or that the content of the volume won't be changed?
Volumes persist until no containers use them
-> What's the definition of 'use'? As long as a container is not stopped, killed, removed? Does that mean that the volume Docker created on the host system will get removed? Or does volume only refer to a virtual bridge between a directory inside Docker and one on the host system?
If you store all your data in the container, what are you going to do when you need to update the image? Updates to images are normally done by changing the Dockerfile and rebuilding the image. If my data is kept separate to my container, I can start a new version of the image, mount the data with --volumes-from or -v and kill the old container. In your case, you have to keep the container running and try to patch in place with something like puppet.
Also, I'm not sure what you think you're saving. If you run the official postgres image, it will have declared volumes in the Dockerfile. Those volumes exist as normal directories on your host system whether you ran the container with -v or not. Even if your Dockerfile has no volumes, clearly the UFS is being stored on your host anyway.
In general, you should consider containers to be temporary and stateless. Whilst you don't have to do this, you will find most of the tooling and support services are designed around this idiom.
Regarding your scenarios, there are a few you're missing:
A bug could make it impossible to restart a stopped container
The updating issue mentioned above
If you want to change storage driver. This will cause a great deal of problems, as you need to migrate your images.
Just for clarity on the commands, docker start will restart stopped or exited containers and docker unpause will unpause paused containers.