Unable to connect mongodb container from python container - mongodb

I am playing around with python docker sdk and my use case is to store data from one container to mongodb container. But whenever my container is running it is throwing an error.
pymongo.errors.ServerSelectionTimeoutError: 172.17.0.4:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 615c9a3b22decbdcf97d4c6b, topology_type: Single, servers: [<ServerDescription ('172.17.0.4', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('172.17.0.4:27017: [Errno 111] Connection refused')>]>
I am trying to build my own image and couldn't notice mongod service in the container as well. I did not find any service file for it as well. Even after triggering "mongod" and also even after changing bingIP to 0.0.0.0, this hasn't worked.
Below is my dockerfile for the mongo
FROM ubuntu:latest
RUN apt-get update && apt-get upgrade
COPY ./UTC /etc/localtime
RUN DEBIAN_FRONTEND=noninteractive
RUN ["apt-get", "install", "-y", "--no-install-recommends", "tzdata", "openssh-server"]
RUN apt-get install -y vim wget apt-utils sudo gnupg
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN useradd -rm -s /bin/bash -g root -G sudo -u 1001 admin
RUN echo "admin:admin" | chpasswd
COPY id_rsa.pub /home/admin/.ssh/authorized_keys
COPY init_script.sh /home
RUN chown -R admin /home/admin/.ssh
RUN chmod 600 /home/admin/.ssh/authorized_keys
RUN wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | apt-key add -
RUN echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-5.0.list
RUN apt-get update
RUN mkdir -p /data/db
USER root
RUN sudo chown -R root /data/db
RUN apt-get install -y --no-install-recommends mongodb-org
EXPOSE 22
EXPOSE 27017
ENTRYPOINT bash -c /home/init_script.sh
Here is my init_script.sh
service ssh restart
mongod >> /dev/null
here is the python code that is trying to reach the mongodb container
def create_db_connect():
info=open('/home/ip_file.json',)
ip_info=json.load(info)
ip=ip_info["/login_mongo"]
url="mongodb://"+ip+":27017/"
myclient = pymongo.MongoClient(url)
return myclient
and the ip_file.json is a dynamically generated file which looks as below
{
"/login_mongo": "172.17.0.3"
}
I am still trying to figure out whether mongo is properly installed or not. Since I am 2 days old in using mongodb, everything is a bit confusing.

I have resolved this issue. Thanks #DavidMaze for the inputs. I can notice that the mongod is only listening to localhost even after binding 0.0.0.0 IP in mongo.conf. The mongodb is not showing as a service and hence we cannot solidly apply those changes even after deleting /tmp/mongo* and mongo locks.
So this is what I did. Instead of executing mongod, I have executed mongod --bind_ip_all. This statement allows to connect all remote hosts.
Such a small logic. I feel like a dumb moron.

Related

pyrhon file in docker container connecting to remote database through vpn

I have the following dockerfile:
# Base image
FROM osgeo/gdal:ubuntu-small-latest
# Working directory
RUN mkdir /code
# Pip and apt-get install packages
RUN apt-get update \
&& apt-get -y install libpq-dev gcc \
&& apt install -y python3-pip
RUN pip install psycopg2
# Copy main file and its binaries -> Run main file
COPY /last_upload.py code/last_upload.py
WORKDIR code
#CMD ["python", "last_upload.py"]
After I build and run the image. I run the following in the opened bash terminal:
python last_upload.py
This file won't run because as some point connection is made with a remote postgres database. The following error is shown:
psycopg2.OperationalError: connection to server at "XXX.XXX.X.X", port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
Should you define a port or something for the docker container?
This ip adress (XXX.XXX.X.X) is approached through vpn.
Anything would help:
ps
On spyder (and not in docker) everything works fine.

Connection To MongoDb running in Docker

I'm really new in using Docker and now I'm trying to run a Mongodb in a Docker locally and get a Connection to it. I don't know why but I can't get any Connection.
Setup: I am using a Mac with MacOS Catalina 10.15.4
- Docker Desktop Version 2.2.0.5
My Dockerfile:
FROM ubuntu
RUN apt-get update && apt-get install -y gnupg2
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' > tee /etc/apt/sources.list.d/mongodb.list
RUN apt-get update
RUN apt-get install -y mongodb
RUN mkdir -p /data/db
EXPOSE 27017
CMD ["--port 27017"]
ENTRYPOINT usr/bin/mongod
after this I run these Commands:
docker build -t testmongodb .
docker run -p 27017:27017 testmongodb
I can see the Image runnig in Docker Desktop
and I try to use Studio 3T to Connect to the Mongodb
my Url for this is: mongodb://localhost:27017/?serverSelectionTimeoutMS=5000&connectTimeoutMS=10000&3t.uriVersion=3&3t.connection.name=localhost%3A27017
The Error I get is:
Connection failed.
SERVER [localhost:27017] (Type: UNKNOWN)
|_/ Connection error (MongoSocketReadException): Prematurely reached end of stream
Details:
Timed out after 5000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
The mongod instance by default only listens on the loopback interface 127.0.0.1. In this case this means the loopback interface inside the container. To also have it listen for external connections you need to specify a separate ip:
ENTRYPOINT usr/bin/mongod --bind_ip 0.0.0.0
As an aside, rolling your own docker image for something like mongodb, which has an official docker image already, is rarely a good idea unless you know what you're doing or need more than the official image can provide.

Mongodb on debian docker image - unable to stand

Good evening.
I'm noobie in docker and try to learn it a little bit. Currently writing simple java application integrated with mongodb, but I stuck on dockerfile. Basically the problem is with mongodb start. Here is my docker file:
FROM debian:buster-slim
# Install necessary libs
RUN apt-get update && apt-get install -y apt-utils wget gnupg gnupg2 curl
# Install mongodb
RUN wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | apt-key add -
RUN echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
RUN systemctl enable mongod.service
RUN service mongod start
# Install jre 11
RUN apt-get install -y openjdk-11-jre
Here is the terminal output (only last step):
Setting up mongodb-org-shell (4.2.1) ...
Setting up mongodb-org-tools (4.2.1) ...
Setting up mongodb-org-mongos (4.2.1) ...
Setting up mongodb-org (4.2.1) ...
Removing intermediate container 7491080bfe9f
---> bbcf5b2ccb13
Step 7/11 : RUN service mongod start
---> Running in 46a66989ade2
mongod: unrecognized service
The command '/bin/sh -c service mongod start' returned a non-zero code: 1
Funny think is that I followed an official mongodb installation guide:
Mongodb installation on debian
During installation on 'real' debian/ubuntu machine it works.
It also doesn't work when tried to build docker image from official mongodb image from docker hub, I mean FROM mongo:4.2-bionic
After login to container and try to run mongo it returns:
root#8cc1d270a262:~# mongo
MongoDB shell version v4.2.0
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
2019-10-23T20:39:44.728+0000 E QUERY [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:341:17
#(connect):2:6
2019-10-23T20:39:44.729+0000 F - [main] exception: connect failed
2019-10-23T20:39:44.729+0000 E - [main] exiting with code 1
I expected that, cause mongo is unable to stand... Somehow.
Any ideas?
So, it seems when trying to follow the instructions to install MongoDB on Debian the SysVInit files are not created and error message mongod: unrecognized service. So a basic question: Does a docker container really need daemon control with either SysVInit or systemd? I don't think it really needs it, and my reason is because the container itself has a single purpose - to host the database. The container should always have the database engine running. With this philosophy in mind, I altered the Dockerfile to include an ENTRYPOINT that starts the mongod instead of relying on any daemon management system.
In order for the MongoDB database to be available outside the container I adjusted the mongod.conf file to bind to all network adapters by using bindIp: 0.0.0.0 instead of bindIp: 127.0.0.1. I also expose port 27017 in the Dockerfile. This means if you have MongoDB installed and running on the host computer using the default port 27107 that process will need to be halted to yield the port to the Docker container.
I was getting some errors in the container around the debconf stuff so I set it non-interactive as well. The installation of java was giving me fits, so I commented it out. If you need java on this container this will still need to be worked out.
Dockerfile:
FROM debian:buster-slim
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install necessary libs
RUN apt-get update && apt-get install -y apt-utils wget gnupg gnupg2 curl
# Install mongodb
RUN wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | apt-key add -
RUN echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
# BIND TO ALL ADAPTERS IN CONTAINER
RUN sed -i "s,\\(^[[:blank:]]*bindIp:\\) .*,\\1 0.0.0.0," /etc/mongod.conf
# Install jre 11
# RUN "apt-get install -y openjdk-11-jre"
EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod", "-f", "/etc/mongod.conf"]
Build:
To build the Docker image issue the following command...
docker build --tag mongodb .
(notice the period in the command - it is required).
Run:
To create a docker container, use the run command.
docker run --publish 27017:27017 --name mongodb -d mongodb
Notice the --publish to map host port 27017 to container port 27017. Notice the --name to name the container for easier reference if we need to get a bash shell inside the container. Run -d for detached mode so it runs in the background, and finally refer to the image named mongodb.
Connect:
Assuming MongoDB is installed on the host too the mongo shell binary will be available. Issue a mongo shell command...
mongo
No other parameters are needed. The installation of MongoDB in the container does not have authorization enabled and so does not ask for a username or password. The default port of 27107 is used by the container and mapped by the docker engine. Localhost is used by default.
Get BASH shell of container:
If you want to get a BASH shell inside of the container issue the following command...
docker exec -it mongodb bash
Try to run mongodb docker container and connect to it using mongo client before building custom images:
docker run --name some-mongo -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -d mongo
docker exec -it some-mongo bash
mongo -u mongoadmin -p secret --authenticationDatabase admin

Starting Postgres in Docker Container

For testing, I'm trying to setup Postgres inside of a docker container so that our python app can run it's test suite against it.
Here's my Dockerfile:
# Set the base image to Ubuntu
FROM ubuntu:16.04
# Update the default application repository sources list
RUN apt-get update && apt-get install -y \
python2.7 \
python-pip \
python-dev \
build-essential \
libpq-dev \
libsasl2-dev \
libffi-dev \
postgresql
USER postgres
RUN /etc/init.d/postgresql start && \
psql -c "CREATE USER circle WITH SUPERUSER PASSWORD 'circle';" && \
createdb -O darwin circle_test
USER root
RUN service postgresql stop && service postgresql start
# Upgrade pip
RUN pip install --upgrade pip
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 5000
# Set the container entrypoint
ENTRYPOINT ["gunicorn", "--config", "/app/config/gunicorn.py", "--access-logfile", "-", "--error-logfile", "-", "app:app"]
When I run:
docker run --entrypoint python darwin:latest -m unittest discover -v -s test
I'm getting:
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
The only way I can get it to work is if I ssh into the container, restart postgres and run the test suite directly.
Is there something I'm missing here?
In a Dockerfile you have
a configuration phase, the RUN directive (and some others)
the process(es) you start, that you put in either
CMD
or
ENTRYPOINT
see the docs
https://docs.docker.com/engine/reference/builder/#cmd
and
https://docs.docker.com/engine/reference/builder/#entrypoint
when a container has completed what it has to do in this start phase, it dies.
This is why the reference Dockerfile for PostgreSQL, at
https://github.com/docker-library/postgres/blob/3d4e5e9f64124b72aa80f80e2635aff0545988c6/9.6/Dockerfile
ends with
CMD ["postgres"]
if you want to start several processes, see supervisord or such tool (s6, daemontools...)
https://docs.docker.com/engine/admin/using_supervisord/

MongoDB, Docker, Meteor: Connection Refused

Meteor works perfectly if I run "meteor". If I setup MongoDB and run Meteor with MONGO_URL set to "mongodb://127.0.0.1:27017/meteor" then it too works perfectly. However, if I run a Docker Container that calls exactly the same Meteor files on the same machine with the MONGO_URL set as above then I get the error: "Exception in callback of async function: Error: failed to connect to [127.0.0.1:27017]". Logic would state that the introduction of Docker is causing the problem. Therefore, is there something I must do to specifically allow Meteor to call MongoDB from inside a container - such as something additional with the MongoDB ports etc.
Dockerfile is:
FROM ubuntu:14.04
MAINTAINER Me "me#me.com"
RUN apt-get update -y && apt-get install --no-install-recommends -y -q chrpath libfreetype6 libfreetype6-dev libssl-dev libfontconfig1
RUN apt-get install --no-install-recommends -y -q build-essential ca-certificates curl git gcc make nano python
ENV PATH /bin:/usr/local/sbin
RUN curl install.meteor.com | sh
ENV ROOT_URL 127.0.0.1
ENV PORT 3000
ENV MONGO_URL mongodb://127.0.0.1:27017/meteor
EXPOSE 3000
CMD [ "meteor" ]
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Meteor is called with the following:
docker run --name meteor-dev -it -p 3000:3000 -v /machine/meteor:/opt/meteor -w /opt/meteor meteor-dev
When you are running a container it creates its own network which is isolated from host network.
So when you are tying to connect to Mongo using "mongodb://127.0.0.1:27017/meteor it searches for MongoDB inside your container.
Instead of using 127.0.0.1 use the host ip addresss or hostname.
Or if your MongoDB is running from a container create a link and use the link to start meteor container. Hope this helps