Description
I have a container that has a mongodb, mongo-express and typesense. I need to set that typesense use data from the mongodb inside the container. How can i do this?
I already used and configured the replica set.
Steps to reproduce
This is my docker compose file:
version: "3.9"
services:
database:
image : 'mongo'
container_name: 'container-name'
environment:
- PUID=1000
- PGID=1000
volumes:
- ./database:/data/db
entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
ports:
- 27017:27017
restart: unless-stopped
mongo-express:
image: mongo-express:0.54
ports:
- "8081:8081"
environment:
ME_CONFIG_MONGODB_SERVER: database
depends_on:
- database
typesense:
image: typesense/typesense:0.21.0
ports:
- "8108:8108"
environment:
TYPESENSE_API_KEY: xyz
TYPESENSE_DATA_DIR: /data/typesense
TYPESENSE_ENABLE_CORS: "true"
volumes:
- ./typesense:/data/typesense
When i call the API colletions/data, i get a response and on the end i see num_documents: 0
Github answer
Related
I have a working monstache deployment using docker for elasticsearch and mongodb synchronization. the configuration file is as shown below:
mongo-url = "mongodb://project-db:27017"
elasticsearch-urls = ["http://es7:9200"]
direct-read-namespaces = ["project.data"]
change-stream-namespaces = ["project.data"]
[logs]
error = "./logs/error.log"
[[mapping]]
namespace = "project.data"
index = "Project"
[[script]]
namespace = "project.data"
path = "./scripts/collection.js"
routing = true
However, I need to add several other databases like 10 of them. mongo-url is a string. Is there a way to add several mongodb sources for indexing?
Monstache seems reasonably light. I'd recommend using multiple dockers in a docker compose setup. Something like so:
version: '3.5'
networks:
proxynet:
name: proxynet
driver: bridge
services:
mongodb:
networks:
- mongodbnet
- proxynet
ports:
- 27017:27017
container_name: mongodb
image: mongo:4.2.17
command: --auth --replSet mongoset
volumes:
- /path/to/docker/mongodb/data:/data/db
restart: unless-stopped
elasticsearch:
container_name: elasticsearch
networks:
- proxynet
ports:
- 9200:9200
- 9300:9300
environment:
- "discovery.type=single-node"
- ELASTIC_PASSWORD=random
volumes:
- /path/to/docker/elasticsearch/data:/usr/share/elasticsearch/data
image: elasticsearch:7.5.2
restart: unless-stopped
monstache01:
networks:
- proxynet
image: rwynn/monstache
container_name: monstache
ports:
- 8080:8080
working_dir: /monstache
command: -f ./monstache.toml
volumes:
- /path/to/monstache01:/monstache
depends_on:
- elasticsearch
- mongodb
restart: unless-stopped
monstache02:
networks:
- proxynet
- mongodbnet
image: rwynn/monstache
container_name: monstache
ports:
- 8081:8080
working_dir: /monstache
command: -f ./monstache.toml
volumes:
- /path/to/docker/monstache02:/monstache
depends_on:
- elasticsearch
- mongodb
restart: unless-stopped
Problem:
I have created a mongo Db replica set like this using docker-compose.
questionsdb:
image: mongo:latest
container_name: questionsdb
hostname: questionsdb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=mi1234
volumes:
- mongo_data:/data/dbadmin
ports:
- 27017:27017
- 9229:9229
entrypoint: [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"]
#networks:
# - custom-
questionsdb1:
image: mongo:latest
container_name: questionsdb1
hostname: questionsdb1
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=mi1234
volumes:
- mongo_data1:/data/dbadmin
ports:
- 27018:27017
- 9230:9229
entrypoint: [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"]
#networks:
# - custom-network
questionsdb2:
image: mongo:latest
container_name: questionsdb2
hostname: questionsdb2
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=mi1234
volumes:
- mongo_data2:/data/dbadmin
expose:
- "27017"
ports:
- 27019:27017
- 9231:9229
entrypoint: [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"]
#networks:
# - custom-network
And my mongo-express container configuration is like this.
mongo-express:
image: mongo-express
container_name: mongo-express
restart: always
ports:
- 8111:8081
environment:
- ME_CONFIG_MONGODB_SERVER=questionsdb
- ME_CONFIG_MONGODB_ADMINUSERNAME=admin
- ME_CONFIG_MONGODB_ADMINPASSWORD=mi1234
- ME_CONFIG_BASICAUTH_USERNAME=admin#mi.com
- ME_CONFIG_BASICAUTH_PASSWORD=mi#1234
#networks:
# - custom-network
And I try to connect to it through mongo Compass like this.
mongodb://admin:mi1234#localhost:27017,localhost:27018,localhost:27019/admin?replicaSet=rsmongo
But both mongo-express and mongo compass was failed with giving me authentication failed.
This is the error I can see in the docker container.
{"t":{"$date":"2021-08-16T01:57:15.391+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn219","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"admin","authenticationDatabase":"admin","remote":"172.18.0.15:59638","extraInfo":{},"error":"UserNotFound: Could not find user \"admin\" for db \"admin\""}}
Can someone help me to solve this. I tried a lot to find out a solution to this but I was unable to do so. Thank you
According to the error message no admin user has been created in your DB during the container startup.
This could be explained by your custom entrypoint [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"] that overrides the official one. Whereas the root user creation work seems to be achieved by this script.
Try to execute your docker-compose stack with only one mongo instance and without the entrypoint statement.
I am having this error while running my docker- compose command "docker-compose up -d mongo". Please help me to find a solution . Thanks in advance .
My docker-compose.yml looks like this .
version: '3'
services:
mongo:
image: mongo:4.2.0
ports:
- "27017:27017"
volumes:
- mongodb:/data/db
- mongodb_config:/data/configdb
mongo-express:
image: mongo-express
restart: unless-stopped
ports:
- "8081:8081"
volumes:
mongo:
external: true
please help .
The error message means you should create the volume first. Try this:
version: '3'
services:
mongo:
image: mongo:4.2.0
ports:
- "27017:27017"
volumes:
- mongodb:/data/db
- mongodb_config:/data/configdb
mongo-express:
image: mongo-express
restart: unless-stopped
ports:
- "8081:8081"
volumes: # use volume by local driver, so you can use the volume directly
mongodb:
driver: local
mongodb_config:
driver: local
See also Use volumes
I am using the following docker-compose.yml for deployment cloned from following https://fusionauth.io/docs/v1/tech/installation-guide/docker
version: '3'
services:
db:
image: postgres:9.6
environment:
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
# Un-comment to access the db service directly
ports:
- 5432:5432
networks:
- db
restart: unless-stopped
volumes:
- db_data:/var/lib/postgresql/data
search:
image: docker.elastic.co/elasticsearch/elasticsearch:6.3.1
environment:
- cluster.name=fusionauth
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=${ES_JAVA_OPTS}"
# Un-comment to access the search service directly
ports:
- 9200:9200
- 9300:9300
networks:
- search
restart: unless-stopped
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- es_data:/usr/share/elasticsearch/data
fusionauth:
image: fusionauth/fusionauth-app:latest
depends_on:
- db
- search
environment:
DATABASE_URL: jdbc:postgresql://db:5432/fusionauth
DATABASE_ROOT_USER: ${POSTGRES_USER}
DATABASE_ROOT_PASSWORD: ${POSTGRES_PASSWORD}
DATABASE_USER: ${DATABASE_USER}
DATABASE_PASSWORD: ${DATABASE_PASSWORD}
FUSIONAUTH_MEMORY: ${FUSIONAUTH_MEMORY}
FUSIONAUTH_SEARCH_SERVERS: http://search:9200
FUSIONAUTH_URL: http://fusionauth:9011
networks:
- db
- search
restart: unless-stopped
ports:
- 9011:9011
volumes:
- fa_config:/usr/local/fusionauth/config
networks:
db:
driver: bridge
search:
driver: bridge
volumes:
db_data:
es_data:
fa_config:
I am unable to access the fusionAuth UI screen from http://localhost:9011 or http://fusionauth:9011
How can I access the UI welcome screen from outside docker container? Is there any
enableExternal: true env variable available for docker-compose.yml?
Is there a way to test or ping the fusionAuth App server to make sure it's up and running other than using docker ps -a
This was due to an error with the docker image. An updated docker image was released hours after this question was asked, and that resolved the issue.
More details here: https://github.com/FusionAuth/fusionauth-containers/issues/47
I have been trying to dockerize my spring boot application which depends on redis, kafka and mongodb.
Following is the docker-compose.yml:
version: '3.3'
services:
my-service:
image: my-service
build:
context: ../../
dockerfile: Dockerfile
restart: always
container_name: my-service
environment:
KAFKA_CONFLUENT_BOOTSTRAP_SERVERS: kafka:9092
MONGO_HOSTS: mongodb:27017
REDIS_HOST: redis
REDIS_PORT: 6379
volumes:
- /private/var/log/my-service/:/var/log/my-service/
ports:
- 8080:8090
- 1053:1053
depends_on:
- redis
- kafka
- mongodb
portainer:
image: portainer/portainer
command: -H unix:///var/run/docker.sock
restart: always
container_name: portainer
ports:
- 9000:9000
- 9001:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
redis:
image: redis
container_name: redis
restart: always
ports:
- 6379:6379
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookeeper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
container_name: kafka
environment:
KAFKA_CREATE_TOPICS: "cms.entity.change:1:1" # topic:partition:replicas
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_PORT: 9092
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- "zookeeper"
mongodb:
image: mongo:latest
container_name: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME:
MONGO_INITDB_ROOT_PASSWORD:
ports:
- 27017:27017
volumes:
- ./data/db:/data/db
The issue is that this starts up mongo as a STANDALONE instance. So the APIs in my service that persist data are failing as mongo needs to start as a REPLICA_SET.
How can I edit my docker-compose file to start mongo as a REPLICA_SET?
I had the same issue and ended up on this stackoverflow post.
We had a requirement of using official mongoDB docker image (https://hub.docker.com/_/mongo ) and couldn't use bitnami as suggested in Vahid's answer.
This answer isn't exactly what's needed by the question asked and coming in 6 months too late; but it should give directions to someone who need to use the mongoDb standalone replicaset throw away instance for integration testing purpose. If you need to use it in PROD then you'll have to provide environment variables for volumes and auth as per Vahid's answer.
version: '3.7'
services:
mongodb:
image: mongo:latest
container_name: myservice-mongodb
networks:
- myServiceNetwork
expose:
- 27017
command: --replSet singleNodeReplSet
mongodb-replicaset:
container_name: mongodb-replicaset-helper
depends_on:
- mongodb
networks:
- myServiceNetwork
image: mongo:latest
command: bash -c "sleep 5 && mongo --host myservice-mongodb --port 27017 --eval \"rs.initiate()\" && sleep 2 && mongo --host myservice-mongodb --port 27017 --eval \"rs.status()\" && sleep infinity"
my-service:
depends_on:
- mongodb-replicaset
image: myserviceimage
container_name: myservicecontainer
networks:
- myServiceNetwork
environment:
myservice__Database__ConnectionString: mongodb://myservice-mongodb:27017/?connect=direct&replicaSet=singleNodeReplSet&readPreference=primary
myservice__Database__Name: myserviceDb
networks:
myServiceNetwork:
driver: bridge
NOTE: Please look at the way how connection string is passed as env variable to the service depending on mongo replicaset instance. You'd have to ensure that the name used in setting up the mongodb replicaset (in my case singleNodeReplicaSet) is passed on to the service depending on it.
Edited:
my previous answer was far wrong so I changed it. I managed to make it work using 'bitnami/mongodb:4.0'. Not sure if that would help you or not, but maybe it gives you some idea. They have a docker-compose file ready for replicaset mode.
version: '3'
services:
mdb-primary:
image: 'bitnami/mongodb:4.0'
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-primary
mdb-secondary:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-secondary
mdb-arbiter:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-arbiter
mongo-cli:
image: 'bitnami/mongodb:latest'
don't forget to add volumes and map it to /bitnami on the primary node
the last container, mongo-cli is for testing purposes. So you can connect to the replicaset using the cli, there is an argument about that here if you like to read about it.
$ docker-compose exec mongo-cli bash
$ mongo "mongodb://mdb-primary:27017/test?replicaSet=replicaset"