In these days, I am trying to deploy my Spring Boot OAuth2 project. It has 3 different modules.(Authentication Server, Resource Server and Front-end)
Authentication and Resource servers have own *.yml file for configurations such as mongodb name-port, server profile-ip etc.
What I am trying to do exactly? I want to deploy spring boot application on docker but i dont want to put my database(mongodb) on docker as a container.
I am not sure this structure is possible or not ?
Because When i run my mongodb on my local(localhost:27017) after that try to deploy spring boot application on local docker as a container, i am getting Timeout exception for MongoDB. The application couldnt connect to external mongoDB(non docker container).
What should I do? Should I run mongodb on docker? I tried it also, Mongo runs successfully but still spring container couldnt run and connect to mongo.
I tried to run another spring boot app without mongodb, it is working successfully and i made request from browser by ip&port, i got response from application as i expected.
*** MONGO URL ****
mongodb://127.0.0.1:27017/db-localhost
**** Authentication server .yml file ****
server:
port: 9080
contextPath: /auth-service
tomcat:
access_log_enabled: true
basedir: target/tomcat
security:
basic:
enabled: false
spring:
profiles:
active: development
thymeleaf:
cache: false
mongo:
db:
server: 127.0.0.1
port: 27017
logging:
level:
org.springframework.security: DEBUG
---
spring:
profiles: development
data:
mongodb:
database: db-localhost
---
spring:
profiles: production
data:
mongodb:
database: db-prod
---
***** DOCKER FILE *******
FROM java:8
VOLUME /tmp
ADD auth-server-1.0-SNAPSHOT.jar app.jar
EXPOSE 9080
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
**** DOCKER COMMAND *******
docker run -it -P --name authserver authserver
The issue with your configuration is referencing the mongodb from inside of the authservice on 127.0.0.1 which is the loopback adapter inside of the authservice container. So you tell your spring application that mongodb is running in the same container as the authservice spring application, which is not the case.
Either you are running your database as an own container (which requires to handle the data volumes correctly) and referencing it using the container name as hostname (via link) or you need to reference the externally running mongodb instance with the correct address. This would be the ip address of the machine running the docker daemon (I assume for your local environment something like 192.168.0.xxx).
Question: What should I do?
At least for developing purposes I would recommend to also use docker for your mongodb instance. I had a similar setup with RabbitMQ in addition and it solved a lot of problems when I used docker for those as well. Using docker-compose to set everything up makes it even easier. Later you can still specify which mongodb instance you want to use through your spring properties.
Problem: I tried it also, Mongo runs successfully but still spring container couldnt run and connect to mongo
The problem is probably because you have not set up any networks or hostnames for you services. Your spring application can not resolve the hostname of your mongo server, since you specified 127.0.0.1 for your mongodb server in your properties.
I would recommend using docker for your mongodb and use a docker-compose.yml file like this to set everything up:
version: '3.7'
services:
resource-server:
image: demo/resource-server:latest
container_name: resource-server
depends_on:
- mongodb-example
networks:
- your-network
ports:
- 8080:8080
auth-server:
image: demo/auth-server:latest
container_name: auth-server
depends_on:
- mongodb-example
networks:
- your-network
ports:
- 8081:8080
mongodb-example:
image: mongo:latest
container_name: mongo-example
hostname: mongo-example
networks:
- your-network
ports:
- 27017:27017
networks:
your-network:
name: network-name
Of course you then need to adapt your property file or specify environment variables through your docker-compose.yml file.
Related
I have testng project with selenium for integration testing of frontend app in vuejs and springboot backend. So in order to run tests I need first to bring up all dependent projects:
springboot and mongodb
vuejs frrontend app
Each project is in its own repo.
So I have created docker images of springboot and frontend app and will put it up in gitlab container registry.
Then in the testeng project plan to use docker-compose in .gitlab-ci.yml. Here is docker-compose.yml for testng project:
version: '3.7'
services:
frontendapp:
image: demo.app-frontend-selenium
container_name: frontend-app-selenium
depends_on:
- demoapi
ports:
- 8080:80
demoapi:
image: demo.app-backend-selenium
container_name: demo-api-selenium
depends_on:
- mongodb
environment:
- SPRING_PROFILES_ACTIVE=prod
- SCOUNT_API_ENDPOINTS_WEB_CORS_OPTIONS_ALLOWEDORIGINS=*
- SPRING_DATA_MONGODB_HOST=mongodb
- SPRING_DATA_MONGODB_DATABASE=demo-api-selenium
- KEYCLOAK_AUTH-SERVER-URL=https://my-keycloak-url/auth
ports:
- 8082:80
mongodb:
image: mongo:4-bionic
container_name: mongodb-selenium
environment:
MONGO_INITDB_DATABASE: demo-api-selenium
ports:
- 27017:27017
volumes:
- ./mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js:ro
After running docker-compose in gitlab-ci.yml what will be url of frontend app in order to execute tests?
When I do it locally I am using following urls for testing:
frontend app: http://localhost:8080
api: http://localhost:8082
But in case when running on gitlab ci what will be url to access frontend and api?
TL;DR instead of using localhost you need to use the hostname of your docker daemon (docker:dind) service. If you setup docker-in-docker for your GitLab job per usual setup, this is most likely docker.
So the urls you need to use according to your compose file are:
frontend app: http://docker:8080
api: http://docker:8082
my_job:
services:
- name: docker:dind
alias: docker # this is the hostname of the daemon
variables:
DOCKER_TLS_CERTDIR: ""
DOCKER_HOST: "tcp://docker:2375"
image: docker:stable
script:
- docker run -d -p 8000:80 strm/helloworld-http
- apk update && apk add curl # install curl and let server start
- curl http://docker:8000 # use the daemon to reach your containers
For a full explanation of this, read on.
Docker port mapping in Gitlab CI vs locally
How it works locally
Normally, when you use docker-compose locally on your system, you are typically running the docker daemon on your localhost (e.g. using docker desktop).
When you provide a port mapping like 8080:80 it means to publish port 8080 on the daemon host bound to port 80 in the container. When running locally, that means you can reach the container via localhost.
In GitLab
However, when you're running docker-in-docker on GitLab CI the important difference in this environment is that the docker daemon is remote. So, when you expose ports through the docker API, the ports are exposed on the docker daemon host not locally in your job container.
Hence, you must use the hostname of the docker daemon, not localhost, to reach your started containers.
Alternative solutions
An alternative to this would be to conduct your testing inside the same docker network that you create with your compose stack. That way, your testing is agnostic of where the docker environment lives and can, for example, leverage the service aliases in your compose file (like frontendapp, demoapi, etc) instead of relying on published ports.
For example, you may choose add a test container to your compose stack. Some testing libraries like Testcontainers can help set this up, too.
I am currently trying to use Docker for my new Django/Postgres project. I am working on a Mac and usually use Postico to quickly connect to my database.
I used to connect like here:
I used the official Docker documentation to setup docker-compose. I now have the issue, that I can't connect via Postico to the postgres db. It seems to me that the problem comes from the ports not being exposed.
version: '3'
services:
db:
image: postgres
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Just map the port to the host machine, add this to the db service in your Compose file:
ports:
- "5432:5432"
Also make sure to set the postgres password variable in the compose file like this
environment:
POSTGRES_PASSWORD: example
The default user is postgres, you can change it with the POSTGRES_USER variable.
You can read about the usage of the image with all options here: https://hub.docker.com/_/postgres/
By default Compose sets up a single network for your app.
Each container can be accessed by the name of the service in the compose file.
In your case you don't have to expose the port to the host machine for your web app to have access to it. You can simply use db as the hostname for postgres (and 5432 for the port) from any other service running on the same compose.
Actually a very similar example is provided in the docker compose documentation:
https://docs.docker.com/compose/networking/
I have built my app which runs fine locally. When I try to run it in docker (docker-compose up) it appears to start, but then throws an error message:
Creating mongodb ... done
Creating webcms ... done
Attaching to mongodb, webcms
...
Mongoose connection "error" event fired with:
MongoError: failed to connect to server [localhost:27017] on first connect
...
webcms exited with code 1
I have read that with Keystone.js you need to configure the Mongo location in the .env file, which I have:
MONGO_URI=mongodb://localhost:27017
Here is my Docker file:
# Use node 9.4.0
FROM node:9.4.0
# Copy source code
COPY . /app
# Change working directory
WORKDIR /app
# Install dependencies
RUN npm install
# Expose API port to the outside
EXPOSE 3000
# Launch application
CMD ["node","keystone"]
...and my docker-compose
version: "2"
services:
# NodeJS app
web:
container_name: webcms
build: .
ports:
- 3000:3000
depends_on:
- mongo
# MongoDB
mongo:
container_name: mongo
image: mongo
volumes:
- ./data:/data/db/mongo
ports:
- 27017:27017
When I run docker ps it confirms that mongo is up and running in a container...
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f3e06e4a5cfe mongo "docker-entrypoint.s…" 2 hours ago Up 2 hours 0.0.0.0:27017->27017/tcp mongodb
I am either missing some config or I have it configured incorrectly. Could someone tell me what that is?
Any help would be appreciated.
Thanks!
It is not working properly because you are sending the wrong host.
your container does not understand what is localhost:27017 since it's your computer address and not its container address.
Important to understand that each service has it's own container with a different IP.
The beauty of the docker-compose that you do not need to know your container address! enough to know your service name:
version: "2"
volumes:
db-data:
driver: local
services:
web:
build: .
ports:
- 3000:3000
depends_on:
- mongo
environment:
- MONGO_URI=mongodb://mongo:27017
mongo:
image: mongo
volumes:
- "db-data:/data/db/mongo"
ports:
- 27017:27017
just run docker-compose up and you are all-set
A couple of things that may help:
First. I am not sure what your error logs look like but buried in my error logs was:
...Error: The cookieSecret config option is required when running Keystone in a production environment.Update your app or environment config so this value is supplied to the Keystone constructor....
To solve this problem, in your Keystone entry file (eg: index.js) make sure your Keystone constructor has the cookieSecret parameter set correctly: process.env.NODE_ENV === 'production'
Next. Change the mongo uri from the one Keystone generated (mongoUri: mongodb://localhost/my-keystone) to: mongoUri: 'mongodb://mongo:27017'. Docker needs this because it is the mongo container address. This change should also be reflected in your docker-compose file under the environment variable under MONGO_URI:
... environment: - MONGO_URI=mongodb://mongo:27017 ...
After these changes your Keystone constructor should look like this:
const keystone = new Keystone({
adapter: new Adapter(adapterConfig),
cookieSecret: process.env.NODE_ENV === 'production',
sessionStore: new MongoStore({ url: 'mongodb://mongo:27017' }),
});
And your docker-compose file, something like this (I used a network instead of links for my docker-compose as Docker has stated that links are a legacy option. I've included mine in case its useful for anyone else):
version: "3.3"
services:
mongo:
image: mongo
networks:
- appNetwork
ports:
- "27017:27017"
environment:
- MONGO_URI=mongodb://mongo:27017
appservice:
build:
context: ./my-app
dockerfile: Dockerfile
networks:
- appNetwork
ports:
- "3000:3000"
networks:
appNetwork:
external: false
It is better to use mongo db atlas if you does not want complications. You can use it in local and in deployment.
Simple steps to get the mongo url is available in https://www.mongodb.com/cloud/atlas
Then add a env variable
CONNECT_TO=mongodb://your_url
For passing the .env to docker, use
docker run --publish 8000:3000 --env-file .env --detach --name kb keystoneblog:1.0
I am having a strange situation where I can not connect to my running mongo DB in my docker compose. My compose file:
version: '3'
services:
app:
image: myimage:latest
ports:
- "8080:8080"
external_links:
- myname:mongo
environment:
- MONGO_URL=mongodb://myname:27017/test
I have found a few infos on that that all did not solve my issue. I.e. I tried:
1)
Create a custom network:
docker network create mongonet
Then start mongo with the --network mongonet flag and add to the compose:
networks:
default:
external:
name: mongonet
Got nothing there either.
I looked into my /etc/hosts file on my compose, and it did not list any DNS entry.
If i do a docker inspect and grab the mongo IP and add it to my compose, that is fine and works like a charm.
I start mongo like this:
docker run -d -p 27017:27017 -v ~/mongo_data:/data/db mongo
I am really rather confused as I believed this to be a out-of-the-box kind of thing. Strangely I can't make it work. I have found examples on internal links (vs external_link) but that does not work for me as I have many services that I would like to run like this and not all of them should run at the same time.
I start my docker compose as this:
docker-compose up --force-recreate
My versions are:
docker-compose version 1.17.1, build 6d101fb
Docker version 17.05.0-ce, build 89658be
My question: How do I successfully link a running mongo container as an external link into my application containers such that they can connect to them?
My docker PS:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5cf6e08d6fde mongo "docker-entrypoint..." About an hour ago Up About an hour 0.0.0.0:27017->27017/tcp gallant_feynman
Links are deprecated, use networks instead.
Notes:
If you’re using the version 2 or above file format, the
externally-created containers must be connected to at least one of the
same networks as the service which is linking to them. Links are a
legacy option. We recommend using networks instead.
The network way should work. I think you are missing some pieces. Make sure to give the mongo container a name, and make sure to attach the app container to the network in the compose file:
docker network create mongonet
docker run -d -p 27017:27017 --network mongonet --name mongo -v ~/mongo_data:/data/db mongo
version: '3'
services:
app:
image: myimage:latest
ports:
- "8080:8080"
environment:
- MONGO_URL=mongodb://mongo:27017/test
networks:
- mongonet
networks:
default:
external:
name: mongonet
I'm using a Dockerfile in combination with a docker-compose.yml to start two services:
My app service
A MongoDB service
My docker-compose.yml:
web:
build: .
ports:
- "80:3000"
environment:
NODE_ENV: production
links:
- mongo
mongo:
image: mongo
command: --smallfiles
ports:
- "27017:27017"
I can't seem to figure out how to control access to the MongoDB container (like with the --auth flag), and how to have external access (say a GUI) using a username/password.
The two services get redeployed via Tutum by a webhook after a Docker Automated Build. In other words, I don't want to manually configure the database every time.
How do I control access a.k.a. set a root/admin user to secure my MongoDB database using the Dockerfile or the docker-compose.yml file?