In a GitLab CI pipeline I need to connect my own C++ code from a docker container to a MongoDB running as servive but I cannot connect.
This is a minimal gitlab-ci.yml example showing the problem:
stages:
- connect1
- connect2
image: docker:latest
variables:
MONGOIMAGE: "mongo:4.2.3-bionic"
MONGO_INITDB_ROOT_USERNAME: "root"
MONGO_INITDB_ROOT_PASSWORD: "geheim"
connect1:
stage: connect1
services:
- name: $MONGOIMAGE
image: mongo
script:
- mongo --host mongo --username $MONGO_INITDB_ROOT_USERNAME --password $MONGO_INITDB_ROOT_PASSWORD --eval "db.help()"
connect2:
stage: connect2
services:
- name: $MONGOIMAGE
script:
- docker run --rm mongo mongo --host mongo --username $MONGO_INITDB_ROOT_USERNAME --password $MONGO_INITDB_ROOT_PASSWORD --eval "db.help()"
The error message of connect2 is that the host cannot be found:
connecting to: mongodb://mongo:27017/?compressors=disabled&gssapiServiceName=mongodb
2020-06-04T16:20:47.614+0000 E QUERY [js] Error: couldn't connect to server mongo:27017, connection attempt failed: HostNotFound: Could not find address for mongo:27017: SocketException: Host not found (authoritative) :
connect#src/mongo/shell/mongo.js:341:17
#(connect):2:6
2020-06-04T16:20:47.615+0000 F - [main] exception: connect failed
2020-06-04T16:20:47.615+0000 E - [main] exiting with code 1
I already tried with --host localhost but this is working neighter. How can I achive that a container started with docker run like in connect2 can connect to the MongoDB service?
Finally I found a solution:
connect2:
stage: connect2
services:
- name: $MONGOIMAGE
script:
- ping mongo -c 5
- docker run --rm mongo mongo --add-host mongo:`cat /etc/hosts | grep mongo | awk '{print $1}'` --net host --username $MONGO_INITDB_ROOT_USERNAME --password $MONGO_INITDB_ROOT_PASSWORD --eval "db.help()"
Docker is client/server application. You're running the client, but there is no server. So you need to run a Docker daemon using Docker-in-Docker.
You can see an example GitLab CI config here: https://stackoverflow.com/a/61106578/6214034
And the relevant documentation: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html
Related
I'm trying to setup mongo on Travis to be used for my integration tests. My travis configuration that initializes mongodb is:
env:
global:
- MONGODB_URI: mongodb://fillrx_test:test#127.0.0.1:27017/fillrx_test_db
before_install:
- sleep 15
- mongo fillrx_test_db --eval 'db.createUser({user:"fillrx_test", pwd:"test",roles:["readWrite"]});'
- docker build -t jeremycod/api-test -f ./server/Dockerfile.dev ./server
script:
- docker run --env MONGODB_URI -e CI=true jeremycod/api-test npm test
However, this fails on Travis with the connection refused error:
1) Create address endpoint /api/v1/address/user/userId
"before all" hook: connectToTestDB for "Create address with correct input":
MongoNetworkError: failed to connect to server [127.0.0.1:27017] on first connect [Error: connect ECONNREFUSED 127.0.0.1:27017
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16) {
name: 'MongoNetworkError'
}]
UPDATE:
Solution to this problem is to run docker container with "--network host", e.g.
- docker run --network host --env MONGODB_URI -e CI=true jeremycod/api-test npm test
This approach binds docker container directly to the Docker host’s network, so it's available from docker container at url:
mongodb://127.0.0.1:27017/mongo_db_name
I have this docker-compose.yml file where I run a mongo container
version: '3'
services:
appapi:
container_name: appapi
image: strapi/strapi:3.1.3
environment:
DATABASE_CLIENT: ${APPAPI_DATABASE_CLIENT}
DATABASE_HOST: ${APPAPI_DATABASE_HOST}
DATABASE_PORT: ${APPAPI_DATABASE_PORT}
DATABASE_NAME: ${APPAPI_DATABASE_NAME}
DATABASE_USERNAME: ${APPAPI_DATABASE_USERNAME}
DATABASE_PASSWORD: ${APPAPI_DATABASE_PASSWORD}
ports:
- 1337:1337
volumes:
- ./app:/srv/app
depends_on:
- appmongo
appmongo:
container_name: appmongo
image: mongo:4.4.0
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: ${APPDB_MONGO_INITDB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${APPDB_MONGO_INITDB_ROOT_PASSWORD}
ports:
- "27027:27017"
volumes:
- ./data/db:/data/db
I want to backup the database running a dump
docker run -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=admin --rm mongo mongodump --host mongoapp:27027 --archive --gzip | cat > ./mongodumps/dump_$(date '+%d-%m-%Y_%H-%M-%S').gz
I tried to modify the previous command but I am not able to connect and do the dump, I am getting
2020-08-15T19:27:04.870+0000 Failed: can't create session: could not connect to server: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: mongoapp:27027, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : dial tcp: lookup mongoapp on 192.168.65.1:53: no such host }, ] }
I was able to dump/restore with the following commands
dump
docker exec defymongo sh -c 'mongodump --archive -u {{mongouser}} -p {{mongopass}}' > ./mongodumps/dump_$(date '+%d-%m-%Y_%H-%M-%S').gz
restore
docker exec -i defymongo sh -c 'mongorestore --archive -u {{mongouser}} -p {{mongopass}}' < ./mongodumps/dump_$(date '+%d-%m-%Y_%H-%M-%S').gz
The difference is here the commands use sh -c to execute mongorestore and pass parameters with authentication values.
This is not enough to backup Strapi. Probably there are some values inside the /src/app folder in Strapi that should also be backed up
Hopefully this will helps someone else
I'm running an app with mongo db with docker.
Here is the mongodb part of the docker-compose file.
mongodb:
restart: always
image: mongo:3.6.3
container_name: "mongodb"
environment:
- MONGO_DATA_DIR=/data/db
- MONGO_LOG_DIR=/dev/nul
- MONGO_INITDB_ROOT_USERNAME=MasterUser
- MONGO_INITDB_ROOT_PASSWORD=MasterPassword
ports:
- 27017:27017
when i start the apps with docker-compose i see that mongodb is not detecting the root username and password therefore i can't login to mongodb and create user for my applications.
Strange thing is that, when i create a compose file only for mongodb it runs and uses the credentials i gave (MasterUser/MasterPassword)
i can also see that master user password and username are set in ENV in the container
docker exec -it mongodb env | grep -i master
MONGO_INITDB_ROOT_USERNAME=MasterUser
MONGO_INITDB_ROOT_PASSWORD=MasterPassword
however they dont work to login to db
docker exec -it mongodb mongo -u MasterUser -p MasterPassword --authenticationDatabase admin
MongoDB shell version v3.6.3
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.3
2020-07-18T19:27:22.378+0000 E QUERY [thread1] Error: Authentication failed. :
#(auth):6:1
#(auth):1:2
exception: login failed
I'm trying to create a docker file that mongo running in a different container, to execute commands from a file, however the second container receives a connection refused error:
mongo-testdata_1 | MongoDB shell version: 3.2.9
mongo-testdata_1 | connecting to: mongo:27017/admin
mongo-testdata_1 | 2019-12-15T15:57:30.067+0000 W NETWORK [thread1] Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
mongo-testdata_1 | 2019-12-15T15:57:30.067+0000 E QUERY [thread1] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed :
mongo-testdata_1 | #./testData.js:1:20
mongo-testdata_1 |
mongo-testdata_1 | failed to load: ./testData.js
How would I be able to connect to the mongoDB running on the first container, using the mongo cli in the second?
commands used
docker-compose up -d mongo
docker exec my-test-mongo mongo --eval "db.createUser({ user: 'admin', pwd: 'password', roles: [ 'dbAdmin', 'readWrite' ] });" admin
docker-compose up --build mongo-testdata
mongo.dockerfile
FROM mongo:3.2.9
ADD testData.js .
ENTRYPOINT ["mongo"]
docker-compose.yml
version: '2'
services:
mongo-testdata:
build:
context: .
dockerfile: mongo.dockerfile
command: --host mongo --port 27017 --username='admin' --password='password' admin ./testData.js
depends_on:
- mongo
networks:
- backend
mongo:
image: mongo:3.2.9
container_name: my-test-mongo
ports:
- 27017:27017
networks:
- backend
networks:
backend:
driver: bridge
testData.js
const myDb = new Mongo().getDB('mydb');
myDb.getCollection('test').insert({
foo: 'bar',
});
You need to specify host and port when you connect to mongo in testData.js.
new Mongo(<host:port>)
In your case this would be new Mongo('mongo').
https://docs.mongodb.com/v3.2/reference/method/Mongo/#Mongo
Instantiation Options
Use the constructor without a parameter to instantiate a connection to the localhost interface on the default port.
Pass the parameter to the constructor to instantiate a connection to the and the default port.
Pass the <:port> parameter to the constructor to instantiate a connection to the and the .
The only depends_on will not work, as the mongo DB container process is not ready to accept to container while CLI get connection refused. You can try two option
- simply put sleep for some second the DB container will ready to accept connection
- Increase wait timeout in mongo shell connection
Here is way to deal with connection refused by adding 10 second seelp in import.
version: '2'
services:
mongo_import:
image: mongo
command: /bin/bash -c "sleep 10 && mongo --host mongo --port 27017 --username='admin' --password='password' admin /root/testData.js"
volumes:
- ./testData.js:/root/testData.js
depends_on:
- mongo
mongo:
image: mongo
container_name: my-test-mongo
ports:
- 27017:27017
Btw the above task you can perform in the same container without the need of any cli container.
Initializing a fresh instance
When a container is started for the first time it will execute files
with extensions .sh and .js that are found in
/docker-entrypoint-initdb.d. Files will be executed in alphabetical
order. .js files will be executed by mongo using the database
specified by the MONGO_INITDB_DATABASE variable, if it is present, or
test otherwise. You may also switch databases within the .js script.
Initializing a fresh instance
I'm running my mongo db with docker-compose
version: '3'
services:
db:
image: mongo:4
ports:
- "27017:27017"
volumes:
- ./storage/mongo:/data/db:rw
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=admin123
command: --smallfiles --bind_ip_all --port 27017
When I do
$> docker-compose exec db bash
I can simply connect to my db
$> mongo -u admin -p admin123
However, when I do the same thing from my host system I cannot connect
(master) ✗ mongo -u admin -p admin123
MongoDB shell version v4.0.3
connecting to: mongodb://127.0.0.1:27017
Implicit session: session { "id" : UUID("8689631e-ac9c-40f8-aa2a-e55f6103224f") }
MongoDB server version: 4.0.9
2019-05-11T17:44:19.181+0200 E QUERY [js] Error: Authentication failed. :
DB.prototype._authOrThrow#src/mongo/shell/db.js:1685:20
#(auth):6:1
#(auth):1:2
exception: login failed
I also tried with --host 0.0.0.0 but same result. Below is the listing of ps
$> docker-compose ps
Name Command State Ports
-----------------------------------------------------------------------------------------------------------------------------------
db_1 docker-entrypoint.sh --sma ... Up 0.0.0.0:27017->27017/tcp
I get the feeling that mongo doesn't accept remote connections, which is why I added command: --smallfiles --bind_ip_all --port 27017 but it didn't help. Any suggestion what I'm doing wrong?