I have the following dockerfile:
# Base image
FROM osgeo/gdal:ubuntu-small-latest
# Working directory
RUN mkdir /code
# Pip and apt-get install packages
RUN apt-get update \
&& apt-get -y install libpq-dev gcc \
&& apt install -y python3-pip
RUN pip install psycopg2
# Copy main file and its binaries -> Run main file
COPY /last_upload.py code/last_upload.py
WORKDIR code
#CMD ["python", "last_upload.py"]
After I build and run the image. I run the following in the opened bash terminal:
python last_upload.py
This file won't run because as some point connection is made with a remote postgres database. The following error is shown:
psycopg2.OperationalError: connection to server at "XXX.XXX.X.X", port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
Should you define a port or something for the docker container?
This ip adress (XXX.XXX.X.X) is approached through vpn.
Anything would help:
ps
On spyder (and not in docker) everything works fine.
Related
Good evening.
I'm noobie in docker and try to learn it a little bit. Currently writing simple java application integrated with mongodb, but I stuck on dockerfile. Basically the problem is with mongodb start. Here is my docker file:
FROM debian:buster-slim
# Install necessary libs
RUN apt-get update && apt-get install -y apt-utils wget gnupg gnupg2 curl
# Install mongodb
RUN wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | apt-key add -
RUN echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
RUN systemctl enable mongod.service
RUN service mongod start
# Install jre 11
RUN apt-get install -y openjdk-11-jre
Here is the terminal output (only last step):
Setting up mongodb-org-shell (4.2.1) ...
Setting up mongodb-org-tools (4.2.1) ...
Setting up mongodb-org-mongos (4.2.1) ...
Setting up mongodb-org (4.2.1) ...
Removing intermediate container 7491080bfe9f
---> bbcf5b2ccb13
Step 7/11 : RUN service mongod start
---> Running in 46a66989ade2
mongod: unrecognized service
The command '/bin/sh -c service mongod start' returned a non-zero code: 1
Funny think is that I followed an official mongodb installation guide:
Mongodb installation on debian
During installation on 'real' debian/ubuntu machine it works.
It also doesn't work when tried to build docker image from official mongodb image from docker hub, I mean FROM mongo:4.2-bionic
After login to container and try to run mongo it returns:
root#8cc1d270a262:~# mongo
MongoDB shell version v4.2.0
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
2019-10-23T20:39:44.728+0000 E QUERY [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:341:17
#(connect):2:6
2019-10-23T20:39:44.729+0000 F - [main] exception: connect failed
2019-10-23T20:39:44.729+0000 E - [main] exiting with code 1
I expected that, cause mongo is unable to stand... Somehow.
Any ideas?
So, it seems when trying to follow the instructions to install MongoDB on Debian the SysVInit files are not created and error message mongod: unrecognized service. So a basic question: Does a docker container really need daemon control with either SysVInit or systemd? I don't think it really needs it, and my reason is because the container itself has a single purpose - to host the database. The container should always have the database engine running. With this philosophy in mind, I altered the Dockerfile to include an ENTRYPOINT that starts the mongod instead of relying on any daemon management system.
In order for the MongoDB database to be available outside the container I adjusted the mongod.conf file to bind to all network adapters by using bindIp: 0.0.0.0 instead of bindIp: 127.0.0.1. I also expose port 27017 in the Dockerfile. This means if you have MongoDB installed and running on the host computer using the default port 27107 that process will need to be halted to yield the port to the Docker container.
I was getting some errors in the container around the debconf stuff so I set it non-interactive as well. The installation of java was giving me fits, so I commented it out. If you need java on this container this will still need to be worked out.
Dockerfile:
FROM debian:buster-slim
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install necessary libs
RUN apt-get update && apt-get install -y apt-utils wget gnupg gnupg2 curl
# Install mongodb
RUN wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | apt-key add -
RUN echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
# BIND TO ALL ADAPTERS IN CONTAINER
RUN sed -i "s,\\(^[[:blank:]]*bindIp:\\) .*,\\1 0.0.0.0," /etc/mongod.conf
# Install jre 11
# RUN "apt-get install -y openjdk-11-jre"
EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod", "-f", "/etc/mongod.conf"]
Build:
To build the Docker image issue the following command...
docker build --tag mongodb .
(notice the period in the command - it is required).
Run:
To create a docker container, use the run command.
docker run --publish 27017:27017 --name mongodb -d mongodb
Notice the --publish to map host port 27017 to container port 27017. Notice the --name to name the container for easier reference if we need to get a bash shell inside the container. Run -d for detached mode so it runs in the background, and finally refer to the image named mongodb.
Connect:
Assuming MongoDB is installed on the host too the mongo shell binary will be available. Issue a mongo shell command...
mongo
No other parameters are needed. The installation of MongoDB in the container does not have authorization enabled and so does not ask for a username or password. The default port of 27107 is used by the container and mapped by the docker engine. Localhost is used by default.
Get BASH shell of container:
If you want to get a BASH shell inside of the container issue the following command...
docker exec -it mongodb bash
Try to run mongodb docker container and connect to it using mongo client before building custom images:
docker run --name some-mongo -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -d mongo
docker exec -it some-mongo bash
mongo -u mongoadmin -p secret --authenticationDatabase admin
I need to create plpythonu extension on the postgres:10.3 image. For this, I extended the image with an installation command:
FROM postgres:10.3
RUN apt update && apt install -y postgresql-plpython-10
Now I need to run a psql command create extension plpythonu;. But at this point the postgres server is not running yet, so when running docker build . I get an error:
Step 3/3 : RUN psql -c "create extension plpythonu;";
---> Running in 037066c120ea
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
How do I fit this command in? And is there a way to do this without copypasting the entire docker-entrypoint.sh from the original postgres image? Maybe there's a way to specify this in docker-compose?
When step-2 is over you Postgres database down, that is why in Step-3 database is not running.
You can combine step-2 and step-3 together.
RUN apt update && \
apt install -y postgresql-plpython-10 && \
psql -c "create extension plpythonu;"
It means && (logical and) if previous commands successfully over then only execute next one.
For testing, I'm trying to setup Postgres inside of a docker container so that our python app can run it's test suite against it.
Here's my Dockerfile:
# Set the base image to Ubuntu
FROM ubuntu:16.04
# Update the default application repository sources list
RUN apt-get update && apt-get install -y \
python2.7 \
python-pip \
python-dev \
build-essential \
libpq-dev \
libsasl2-dev \
libffi-dev \
postgresql
USER postgres
RUN /etc/init.d/postgresql start && \
psql -c "CREATE USER circle WITH SUPERUSER PASSWORD 'circle';" && \
createdb -O darwin circle_test
USER root
RUN service postgresql stop && service postgresql start
# Upgrade pip
RUN pip install --upgrade pip
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 5000
# Set the container entrypoint
ENTRYPOINT ["gunicorn", "--config", "/app/config/gunicorn.py", "--access-logfile", "-", "--error-logfile", "-", "app:app"]
When I run:
docker run --entrypoint python darwin:latest -m unittest discover -v -s test
I'm getting:
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
The only way I can get it to work is if I ssh into the container, restart postgres and run the test suite directly.
Is there something I'm missing here?
In a Dockerfile you have
a configuration phase, the RUN directive (and some others)
the process(es) you start, that you put in either
CMD
or
ENTRYPOINT
see the docs
https://docs.docker.com/engine/reference/builder/#cmd
and
https://docs.docker.com/engine/reference/builder/#entrypoint
when a container has completed what it has to do in this start phase, it dies.
This is why the reference Dockerfile for PostgreSQL, at
https://github.com/docker-library/postgres/blob/3d4e5e9f64124b72aa80f80e2635aff0545988c6/9.6/Dockerfile
ends with
CMD ["postgres"]
if you want to start several processes, see supervisord or such tool (s6, daemontools...)
https://docs.docker.com/engine/admin/using_supervisord/
I have installed PostgreSQL 9.6.1 on a remote Debian 8.1 minimal VM. I am attempting to install pgAdmin4 in server mode so that I am able to access remotely via the web. I have successfully installed pgAdmin4 within a python virtual environment, but an issue arises at one of the final steps of configuration:
Starting pgAdmin 4. Please navigate to http://localhost:5050 in your browser.
Since I do not have a desktop environment installed (nor do I intend on installing one), how can I complete configuration without using localhost? I have attempted to connect using the server's public IP (e.g. http://80.254.0.132:5050) but am not able to resolve.
I do not have a firewall at the VM or server/NAT level.
I have updated /etc/postgresql/9.6/main/pg_hba.conf and added host all all 0.0.0.0/0 md5 to the configuration.
I have updated /etc/postgresql/9.6/main/postgresql.conf and changed listen_addresses = '*'.
My full steps involved in post-Debian installation (sans new users) looks like the following:
# Initial update.
apt-get install sudo
sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get dist-upgrade -y
sudo apt-get install vim -y
# Postgres.
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'
wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O - | sudo apt-key add -
# Install PostgreSQL.
sudo apt-get update
sudo apt-get install postgresql postgresql-contrib libpq-dev python-dev
# Set postgres password.
sudo -s
cd ~
sudo -u postgres psql postgres
\password postgres
# Allow remote connections.
sudo vim /etc/postgresql/9.6/main/pg_hba.conf
# host all all 0.0.0.0/0 md5
sudo vim /etc/postgresql/9.6/main/postgresql.conf
# listen_addresses = '*'
sudo service postgresql restart
# Python and pgAdmin.
sudo easy_install pip
sudo pip install virtualenvwrapper
# Create the virtual environment and install pgAdmin.
virtualenv pgadmin4
cd pgadmin4
source bin/activate
sudo apt-get install build-essential libssl-dev libffi-dev python-dev libgmp3-dev
sudo pip install cryptography pyopenssl ndg-httpsclient pyasn1
wget https://ftp.postgresql.org/pub/pgadmin3/pgadmin4/v1.1/pip/pgadmin4-1.1-py2-none-any.whl
pip install pgadmin4-1.1-py2-none-any.whl
cp ./lib/python2.7/site-packages/pgadmin4/config.py ./lib/python2.7/site-packages/pgadmin4/config_local.py
python ./lib/python2.7/site-packages/pgadmin4/pgAdmin4.py
By default pgAdmin4 runs on loopback adapter, to make it run on ethernet (eth0) you need to change some of configuration options.
You need to add below config options,
DEFAULT_SERVER = '0.0.0.0'
in config_local.py (in "pgAdmin4" folder).
If also want to change default port then also add
DEFAULT_SERVER_PORT = 8081
Now restart pgAdmin4, Now try accessing pgAdmin4 using IP address you mentioned(eg, http://80.254.0.132:8081), It should work.
Meteor works perfectly if I run "meteor". If I setup MongoDB and run Meteor with MONGO_URL set to "mongodb://127.0.0.1:27017/meteor" then it too works perfectly. However, if I run a Docker Container that calls exactly the same Meteor files on the same machine with the MONGO_URL set as above then I get the error: "Exception in callback of async function: Error: failed to connect to [127.0.0.1:27017]". Logic would state that the introduction of Docker is causing the problem. Therefore, is there something I must do to specifically allow Meteor to call MongoDB from inside a container - such as something additional with the MongoDB ports etc.
Dockerfile is:
FROM ubuntu:14.04
MAINTAINER Me "me#me.com"
RUN apt-get update -y && apt-get install --no-install-recommends -y -q chrpath libfreetype6 libfreetype6-dev libssl-dev libfontconfig1
RUN apt-get install --no-install-recommends -y -q build-essential ca-certificates curl git gcc make nano python
ENV PATH /bin:/usr/local/sbin
RUN curl install.meteor.com | sh
ENV ROOT_URL 127.0.0.1
ENV PORT 3000
ENV MONGO_URL mongodb://127.0.0.1:27017/meteor
EXPOSE 3000
CMD [ "meteor" ]
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Meteor is called with the following:
docker run --name meteor-dev -it -p 3000:3000 -v /machine/meteor:/opt/meteor -w /opt/meteor meteor-dev
When you are running a container it creates its own network which is isolated from host network.
So when you are tying to connect to Mongo using "mongodb://127.0.0.1:27017/meteor it searches for MongoDB inside your container.
Instead of using 127.0.0.1 use the host ip addresss or hostname.
Or if your MongoDB is running from a container create a link and use the link to start meteor container. Hope this helps