I have a few config file that have to be mapped to files inside the container. I want to be able to change these config files on the host and that should reflect in the container. These are basically connection string files that I want to swap without having to rebuild the containers. What I have in my docker-compose.yml is:
services:
portal:
container_name: portal
image: portal
build:
context: .
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./:/var/www/portal
- type: volume
source: ./local/parameters.local.yml
target: /var/www/portal/s/config/parameters.yml
- type: volume
source: ./portal.conf
target: /etc/apache2/sites-available/portal.conf
- awscreds:/root/.aws:ro
I fail to get this to work... I saw some examples where they did not supply the type (or instead of volume they made it "bind") but nothing seems to work for me.
If I build the images with docker compose up and then do docker inspect portal I can see that it has: "Mounts": []
My final plan is to have a docker-compose.yml that has a service called portal and mounts 2 or more files inside the container(NOT copy so that I can change it on my host at will) as well as a few directories. What is kicking me in the face is the files that have to be mapped into the container.
I think you need to change type: volume to type: mount
services:
portal:
container_name: portal
image: portal
build:
context: .
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./:/var/www/portal
- type: mount
source: ./local/parameters.local.yml
target: /var/www/portal/s/config/parameters.yml
- type: mount
source: ./portal.conf
target: /etc/apache2/sites-available/portal.conf
- awscreds:/root/.aws:ro
Also, you can add read-only: true to both of those mounts if you don't want the services to be able to modify parameters.yml or portal.conf.
Just mapping should do the job if the files and folders in the lhs exists in your local machine:
services:
portal:
container_name: portal
image: portal
build:
context: .
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./:/var/www/portal
- ./local/parameters.local.yml:/var/www/portal/s/config/parameters.yml
- ./portal.conf:/etc/apache2/sites-available/portal.conf
- awscreds:/root/.aws:ro
volumes:
awscreds:
Related
I working with a simple docker-compose file (node alpine), i got three anon volumens, this already work in the pass but now, not longer created.
I delete the folder from the host side (Windows), to try if docker creates against the folders and put inside the files, but nothing is happend.
version: "3.3"
services:
api:
#restart: always
build:
context: .
image: foo-foo-platform:1.1.0.0
#container_name: foo-foo-platform
env_file: docker-compose-debug.env
labels:
- "traefik.enable=false"
- "traefik.http.routers.api-gw.rule=PathPrefix(`/`)"
- "traefik.http.services.api-gw.loadbalancer.server.port=8090"
networks:
- internal
volumes:
- /mnt/logs:/mnt/logs
- /mnt/cc:/mnt/cc
ports:
- "8084:8084"
networks:
internal:
I have tried to prune volumes with docker volume prune, anyway noone of volumes listed is from this docker.
Al tried "docker-compose -f docker-compose-debug.yml up --build --force-recreate --renew-anon-volumes"
Note: "/mnt/logs:/mnt/logs" this notation works in windows.
This is my Docker compose/stack file. When I deploy on a single node, everything works fine, but when I deploy on multiple nodes I get the following error:
invalid mount config for type bind bind source path does not exist
version: '3'
services:
shinyproxy:
build: /etc/shinyproxy
deploy:
replicas: 3
user: root:root
hostname: shinyproxy
image: shinyproxy-example
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 5000:5000
networks:
- proxynetwork
mysql:
image: mysql
deploy:
replicas: 3
volumes:
- /mysqldata:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root_password
MYSQL_DATABASE: keycloak
MYSQL_USER: keycloak
MYSQL_PASSWORD: password
networks:
- proxynetwork
keycloak:
deploy:
replicas: 3
image: jboss/keycloak
volumes:
- /etc/letsencrypt/live/ds-gym.de/fullchain.pem:/etc/x509/https/tls.crt
- /etc/letsencrypt/live/ds-gym.de/privkey.pem:/etc/x509/https/tls.key
#- /theme/govuk-social-providers/:/opt/jboss/keycloak/themes/govuk-social-providers/
environment:
- PROXY_ADDRESS_FORWARDING=true
- KEYCLOAK_USER=myadmin
- KEYCLOAK_PASSWORD=mypassword
ports:
- 8443:8443
networks:
- proxynetwork
networks:
proxynetwork:
external: true
I understand that the volumes path is expected on every other node too, but I think this is a very bad practice and my other 2 nodes are anyway just workers. How can I solve that problem? Hopefully there is a solution which allows me to keep the volumes, since I use the same file for docker-compose build to build my images.
Can someone help me?
Thank you :-)
If it is possible you could restrict this service to the node that has the required host path's using placement constraints. However I'm guessing that that's not an option in this use case.
Host mounted volumes should really not be used in a swarm deployment as it would cause redundant data in the filesystems between the nodes. (All files need to be present on all nodes).
One solution would be to implement NFS volumes:
volumes:
example:
driver_opts:
type: "nfs"
o: "addr=<NFS_SERVER_IP>,nolock,soft,rw"
device: ":/docker/path/to/configs"
This solution requires you to host a NFS-Server though. Also keep in mind that this approach is fine for configs but should not be used for file systems that need to provide high performance access.
Regarding your question about keeping your docker-compose file the same across environments: While it is technically possible to do so, most modern projects consist of a base compose file as well as an environment specific override for volumes,networks,images etc.
In a swarm your services will be deployed randomly on your available nodes.
I suppose your "to be mounted directory" is on the manager node, so deploy the wanted service on the manager node like so.
deploy:
placement:
constraints:
- node.role == manager
I am facing some issue in Docker which is installed on EC2 node in AWS.
I have installed ELK in docker using docker-compose and now able to see the logs using tcp filter (winston3-npm) . I have also attached one EBS volume in this EC2 instance , now i want to persist the logs in this EBS so that even if i terminate my EC2 instance and spawn a new instance using this EBS volume then i want to see all the old logs.
So, I am not able to mount a EBS volume to docker so that all my data can be preserved.
Below is my docker-compose file .
Could anyone help me on this ?
version: '3.2'
services:
elasticsearch:
build:
context: elasticsearch/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- type: bind
source: ./elasticsearch/config/elasticsearch.yml
target: /usr/share/elasticsearch/config/elasticsearch.yml
read_only: true
volumes:
- type: bind
source: ./elasticsearch/config/elasticsearch.yml
target: /usr/share/elasticsearch/config/elasticsearch.yml
read_only: true
- /data:/usr/share/elasticsearch/data/:rw
#- type: volume
#source: elasticsearch
#target: /usr/share/elasticsearch/data
ports:
- "9200:9200"
- "9300:9300"
environment:
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
ELASTIC_PASSWORD: changeme
networks:
- elk
logstash:
build:
context: logstash/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- type: bind
source: ./logstash/config/logstash.yml
target: /usr/share/logstash/config/logstash.yml
read_only: true
- type: bind
source: ./logstash/pipeline
target: /usr/share/logstash/pipeline
read_only: true
ports:
- "5000:5000"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
kibana:
build:
context: kibana/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- type: bind
source: ./kibana/config/kibana.yml
target: /usr/share/kibana/config/kibana.yml
read_only: true
ports:
- "5601:5601"
networks:
- elk
depends_on:
- elasticsearch
networks:
elk:
driver: bridge
volumes:
elasticsearch:
EBS is attached to your EC2 instance and there is no direct way to mount your EBS volume in docker container running on your EC2 instance.
You can mount EBS volume on your EC2 instance and point docker persistence storage to that mount point.
I am able to resolve the issue...Have mentioned the EBS mounted directory path in docker.service file which exists inside the /lib/systemd/system/ and able to see all the docker respective data in ebs.
Thank you all for helping me .
I am using docker-compose file generated by docker-app
docker-app render | docker-compose -f - up
The docker app file looks like this and it works as expected. But I am not able to use volumes.
I use -v parameter while using docker run command like this...
-v /my/custom3399:/etc/mysql/conf.d
-v /storage/mysql/datadir3399:/var/lib/mysql
How do I declare volumes in compose file?
# vi hello.dockerapp
# This section contains your application metadata.
# Version of the application
version: 0.1.0
# Name of the application
name: hello
# A short description of the application
description:
# Namespace to use when pushing to a registry. This is typically your Hub username.
#namespace: myHubUsername
# List of application maitainers with name and email for each
maintainers:
- name: root
email:
# Specify false here if your application doesn't support Swarm or Kubernetes
targets:
swarm: false
kubernetes: false
--
# This section contains the Compose file that describes your application services.
version: "3.5"
services:
mysql:
image: ${mysql.image.version}
environment:
MYSQL_ROOT_PASSWORD: india${mysql.port}
ports:
- "${mysql.port}:3306"
--
# This section contains the default values for your application settings.
mysql.image.version: shantanuo/mysql:5.7
mysql.port: 3391
update:
The script mentioned above works well. But once I add volumes, I get an error:
version: "3.5"
services:
mysql:
image: ${mysql.image.version}
environment:
MYSQL_ROOT_PASSWORD: india${mysql.port}
ports:
- "${mysql.port}:3306"
volumes:
- type: volume
source: mysql_data
target: /var/lib/mysql
volumes:
mysql_data:
external: true
And the error is:
docker-app render | docker-compose -f - up
Recreating e5c833e2187d_hashi_mysql_1 ... error
ERROR: for e5c833e2187d_hashi_mysql_1 Cannot create container for service mysql: Duplicate mount point: /var/lib/mysql
ERROR: for mysql Cannot create container for service mysql: Duplicate mount point: /var/lib/mysql
ERROR: Encountered errors while bringing up the project.
As mentioned in the comment, the rendered output is as follows:
# /usr/local/bin/docker-app render
version: "3.5"
services:
mysql:
environment:
MYSQL_ROOT_PASSWORD: india3391
image: shantanuo/mysql:5.7
ports:
- mode: ingress
target: 3306
published: 3391
protocol: tcp
volumes:
- type: volume
source: mysql_data
target: /var/lib/mysql
volumes:
mysql_data:
name: mysql_data
external: true
This issue was resolved once I changed the directory name.
# cd ..
# mv hashi/ hashi123/
# cd hashi123
Not sure how this worked. But since I am able to start the server stack, I am posting it as answer.
Probably simple, but I dont find any thing about this. My composer config file (version 3), define two volumes to be shared with others services :
version: "3"
services:
nginx:
build: docker/nginx
ports:
- "80:80"
volumes:
- config:/etc/nginx/conf.d
- data:/var/http
networks:
- default
container_name: nginx
networks:
default:
volumes:
config:
data:
How to set/attach the local directories (ex. d:/nginx/etc, d:/nginx/http) mapped to this volumes on the config file (or the docker-compose up command) ?
You may try to replace your volumes lines this way :
data:/var/http -> path/to/local/dir:/var/http
Moreover assuming that you are running Windows, it should look like this :
- //d/nginx/etc:/etc/nginx/conf.d
- //d/nginx/http:/var/http
Then remove the global volumes section.