I have created a Spring Cloud microservice ecosystem which is made up of a spring cloud eureka server, a spring cloud gateway proxy and several microservices. I have also used docker to containerize each of this services and i start up the images using a docker-compose file.
version: '3'
services:
discovery-server:
image: serviceregistry-api-docker:latest
ports:
- 8761:8761
networks:
- transaction-network
api-gateway:
image: apigateway-api-docker:latest
environment:
- JAVA_OPTS=
-DEUREKA_SERVER=http://discovery-server:8761/eureka/
depends_on:
- discovery-server
ports:
- 9091:9091
networks:
- transaction-network
utility-service:
image: utility-api-docker:latest
environment:
- JAVA_OPTS=
-DEUREKA_SERVER=http://discovery-server:8761/eureka/
restart: on-failure
depends_on:
- discovery-server
- api-gateway
ports:
- 8090:8090
networks:
- transaction-network
banktransfer-service:
image: banktransfer-api-docker:latest
environment:
- JAVA_OPTS=
-DEUREKA_SERVER=http://discovery-server:8761/eureka/
restart: on-failure
depends_on:
- discovery-server
- api-gateway
ports:
- 8091:8091
networks:
- transaction-network
ussd-service:
image: ussd-api-docker:latest
environment:
- JAVA_OPTS=
-DEUREKA_SERVER=http://discovery-server:8761/eureka/
restart: on-failure
depends_on:
- discovery-server
- api-gateway
ports:
- 8096:8096
networks:
- transaction-network
networks:
transaction-network:
driver: bridge
When i make an update to just 1 microservice and create a new docker image that has my update, how do i replace it with the current running version in the ecosystem ?
Just start it with --no-deps param (even if it already started). For example:
docker-compose up -d --no-deps api-gateway
I used this command to make it possible
$docker-compose up -d --no-deps --build <service_name>
Related
I'm trying to setup a local development with Docker Compose that has a MongoDB cluster as the database. I chose Mongo Express as the Database Admin User Interface so I can check inside the MongoDB database. It does take some time for the cluster to accept connections, I have the 3 db containers as part of the depends_on, but seems like I have to do more than that based on the Docker Compose documentation here. I can't seem to find a good example for waiting for MongoDB clusters. Has anyone figured this out already? Please share, that would be great. Thank you in advance!
Here's the docker-compose.yml file:
version: '3.9'
services:
mongodb-primary:
image: 'bitnami/mongodb:latest'
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-primary
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=password
- MONGODB_REPLICA_SET_KEY=replicasetkey
volumes:
- 'mongodb_master_data:/bitnami'
mongodb-secondary:
image: 'bitnami/mongodb:latest'
depends_on:
- mongodb-primary
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-secondary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_INITIAL_PRIMARY_HOST=mongodb-primary
- MONGODB_INITIAL_PRIMARY_PORT_NUMBER=27017
- MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password
- MONGODB_REPLICA_SET_KEY=replicasetkey
mongodb-arbiter:
image: 'bitnami/mongodb:latest'
depends_on:
- mongodb-primary
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-arbiter
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_INITIAL_PRIMARY_HOST=mongodb-primary
- MONGODB_INITIAL_PRIMARY_PORT_NUMBER=27017
- MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password
- MONGODB_REPLICA_SET_KEY=replicasetkey
dbadmin:
image: mongo-express
restart: always
ports:
- 8081:8081
depends_on:
- mongodb-primary
- mongodb-secondary
- mongodb-arbiter
environment:
ME_CONFIG_MONGODB_URL: mongodb://root:password#mongodb-primary:27017,mongodb-secondary:27017,mongodb-arbiter:27017?replicaSet=replicaset
ME_CONFIG_BASICAUTH_USERNAME: admin
ME_CONFIG_BASICAUTH_PASSWORD: mexpress
volumes:
mongodb_master_data:
driver: local
I have tried many configurations and scenarios based around this which is mostly a tutorial that stops at one ghost instance. I am trying to scale it to 2 with docker-deploy up -d --scale ghost=2. When I hit the individual IP;s of the ghost containers , they work but port 80 is 503.
version: "3.1"
volumes:
mysql-volume:
ghost-volume:
networks:
ghost-network:
services:
mysql:
image: mysql:5.7
container_name: mysql
volumes:
- mysql-volume:/var/lib/mysql
networks:
- ghost-network
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: db
MYSQL_USER: blog-user
MYSQL_PASSWORD: supersecret
ghost:
build: ./ghost
image: laminar/ghost:3.0
volumes:
- ghost-volume:/var/lib/ghost/content
networks:
- ghost-network
restart: always
ports:
- "2368"
environment:
database__client: mysql
database__connection__host: mysql
database__connection__user: blog-user
database__connection__password: supersecret
database__connection__database: db
depends_on:
- mysql
entrypoint: ["wait-for-it.sh", "mysql", "--", "docker-entrypoint.sh"]
command: ["node", "current/index.js"]
haproxy:
image: eeacms/haproxy
depends_on:
- ghost
ports:
- "80:5000"
- "1936:1936"
environment:
BACKENDS: "ghost"
DNS_ENABLED: "true"
LOG_LEVEL: "info"
What I get on localhost:80 is a 503 error the particular eeacms/haproxy image is supposed to be self-configuring any help appreciated
I needed to add a backend URL to the environment and also tell ghost it was installed in an alternate location by adding URL: localhost:5050
I have a implemented a microservice architecture with several servers and databases. I have installed elasticsearch with docker and when I do docker-compose up, everything seems to run fine.
However I would like to integrate the elasticsearch with the several databases (2 mongodb in this sample below) in the system. How do I synch the two mongodb in two different containers with elasticsearch so that I can search them?
client:
container_name: client
stdin_open: true
build:
context: ./client
dockerfile: Dockerfile
restart: always
volumes:
- './client:/app'
ports:
- '1000:3000'
environment:
- NODE_ENV=development
- CHOKIDAR_USEPOLLING=true
weatherdb:
container_name: weather-db
image: mongo
restart: always
ports:
- '2002:27017'
volumes:
- ./weather_service/weather_db:/data/db
networks:
- backend
weather-service:
container_name: weather-service
build: ./weather_service
restart: always
ports:
- "1002:3000"
depends_on:
- weatherdb
links:
- elasticsearch
networks:
- backend
newsdb:
container_name: news-db
image: mongo
restart: always
ports:
- '2003:27017'
volumes:
- ./news_service/news_db:/data/db
networks:
- backend
news-service:
container_name: news-service
build: ./news_service
restart: always
ports:
- "1003:3000"
depends_on:
- newsdb
links:
- elasticsearch
networks:
- backend
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0
container_name: elasticsearch
restart: always
ports:
- 9200:9200
- 9300:9300
environment:
ES_JAVA_OPTS: '-Xms512m -Xmx512m'
network.bind_host: 0.0.0.0
network.host: 0.0.0.0
discovery.type: single-node
volumes:
- ./elasticsearch/esdata:/usr/share/elasticsearch/data
networks:
- backend
Its very simple to just add a elasticsearch docker section in any docker-compose file and start it, all these are independent docker containers and as long as their exposed port on host is not interfering each other and you have the correct configuration in place it should work.
Please refer elasticsearch multi-docker installation using docker file for more info.
NOTE: You have not mentioned what exact issue you are facing, you have mentioned everything ie all docker containers are running file, so please explain in detail what exactly you are trying to solve
I have been trying to dockerize my spring boot application which depends on redis, kafka and mongodb.
Following is the docker-compose.yml:
version: '3.3'
services:
my-service:
image: my-service
build:
context: ../../
dockerfile: Dockerfile
restart: always
container_name: my-service
environment:
KAFKA_CONFLUENT_BOOTSTRAP_SERVERS: kafka:9092
MONGO_HOSTS: mongodb:27017
REDIS_HOST: redis
REDIS_PORT: 6379
volumes:
- /private/var/log/my-service/:/var/log/my-service/
ports:
- 8080:8090
- 1053:1053
depends_on:
- redis
- kafka
- mongodb
portainer:
image: portainer/portainer
command: -H unix:///var/run/docker.sock
restart: always
container_name: portainer
ports:
- 9000:9000
- 9001:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
redis:
image: redis
container_name: redis
restart: always
ports:
- 6379:6379
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookeeper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
container_name: kafka
environment:
KAFKA_CREATE_TOPICS: "cms.entity.change:1:1" # topic:partition:replicas
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_PORT: 9092
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- "zookeeper"
mongodb:
image: mongo:latest
container_name: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME:
MONGO_INITDB_ROOT_PASSWORD:
ports:
- 27017:27017
volumes:
- ./data/db:/data/db
The issue is that this starts up mongo as a STANDALONE instance. So the APIs in my service that persist data are failing as mongo needs to start as a REPLICA_SET.
How can I edit my docker-compose file to start mongo as a REPLICA_SET?
I had the same issue and ended up on this stackoverflow post.
We had a requirement of using official mongoDB docker image (https://hub.docker.com/_/mongo ) and couldn't use bitnami as suggested in Vahid's answer.
This answer isn't exactly what's needed by the question asked and coming in 6 months too late; but it should give directions to someone who need to use the mongoDb standalone replicaset throw away instance for integration testing purpose. If you need to use it in PROD then you'll have to provide environment variables for volumes and auth as per Vahid's answer.
version: '3.7'
services:
mongodb:
image: mongo:latest
container_name: myservice-mongodb
networks:
- myServiceNetwork
expose:
- 27017
command: --replSet singleNodeReplSet
mongodb-replicaset:
container_name: mongodb-replicaset-helper
depends_on:
- mongodb
networks:
- myServiceNetwork
image: mongo:latest
command: bash -c "sleep 5 && mongo --host myservice-mongodb --port 27017 --eval \"rs.initiate()\" && sleep 2 && mongo --host myservice-mongodb --port 27017 --eval \"rs.status()\" && sleep infinity"
my-service:
depends_on:
- mongodb-replicaset
image: myserviceimage
container_name: myservicecontainer
networks:
- myServiceNetwork
environment:
myservice__Database__ConnectionString: mongodb://myservice-mongodb:27017/?connect=direct&replicaSet=singleNodeReplSet&readPreference=primary
myservice__Database__Name: myserviceDb
networks:
myServiceNetwork:
driver: bridge
NOTE: Please look at the way how connection string is passed as env variable to the service depending on mongo replicaset instance. You'd have to ensure that the name used in setting up the mongodb replicaset (in my case singleNodeReplicaSet) is passed on to the service depending on it.
Edited:
my previous answer was far wrong so I changed it. I managed to make it work using 'bitnami/mongodb:4.0'. Not sure if that would help you or not, but maybe it gives you some idea. They have a docker-compose file ready for replicaset mode.
version: '3'
services:
mdb-primary:
image: 'bitnami/mongodb:4.0'
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-primary
mdb-secondary:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-secondary
mdb-arbiter:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-arbiter
mongo-cli:
image: 'bitnami/mongodb:latest'
don't forget to add volumes and map it to /bitnami on the primary node
the last container, mongo-cli is for testing purposes. So you can connect to the replicaset using the cli, there is an argument about that here if you like to read about it.
$ docker-compose exec mongo-cli bash
$ mongo "mongodb://mdb-primary:27017/test?replicaSet=replicaset"
I want to make a developer offline space to develop my database with hasura
I know the existence of the container with the tag cli-migrations, but the command:
hasura-cli console
doesn't work for accessing outside of the container.
My configuration for the docker-compose.yml is:
version: '3'
services:
hasura:
environment:
- HASURA_GRAPHQL_DATABASE_URL=postgres://[some pass]:[some user]#db:5432/[some db]
- HASURA_GRAPHQL_ENABLE_CONSOLE=false
image: hasura/graphql-engine:v1.0.0-rc.1.cli-migrations
container_name: hasura
volumes:
- ./hasura-migrations:/hasura-migrations
networks:
- hasura-db
ports:
- "8081:8080"
- "8082:8081"
restart: always
command: hasura-cli console --console-port 8081 --no-browser
db:
environment:
- POSTGRES_USER=[some user]
- POSTGRES_PASSWORD=[some pass]
- POSTGRES_DB=[some db]
image: postgres:11.4-alpine
container_name: db
restart: always
networks:
- hasura-db
networks:
hasura-db:
There is a Pull request in the hasura graphql project for this issue, but is not merged.
I'm looking for a workaround for this pull request now.
I found a workaround for this!
If I install hasura cli in my machine and use
hasura console --console-port 8080 --endpoint http://127.0.0.1:8081
I can connect to the hasura api and run a console locally.
this is my updated docker-compose.yml
version: '3'
services:
hasura:
environment:
- HASURA_GRAPHQL_DATABASE_URL=postgres://[some pass]:[some user]#db:5432/[some db]
- HASURA_GRAPHQL_ENABLE_CONSOLE=false
image: hasura/graphql-engine:v1.0.0-rc.1.cli-migrations
container_name: hasura
volumes:
- ./hasura-migrations:/hasura-migrations
networks:
- hasura-db
ports:
- "8081:8080"
restart: always
db:
environment:
- POSTGRES_USER=[some user]
- POSTGRES_PASSWORD=[some pass]
- POSTGRES_DB=[some db]
image: postgres:11.4-alpine
container_name: db
restart: always
networks:
- hasura-db
networks:
hasura-db: