I am trying to start my docker-compose.yml (example below), but whenever I start the containers the sshd server service not working:
# My docker-compose.yml
version: '3'
services:
server1:
image: server-dev:v0.8
hostname: server-dev1
command: bash -c "/usr/sbin/init"
ports:
- "2222:22"
- 80:80
server2:
image: server-dev:v0.8
hostname: server-dev2
command: bash -c "/usr/sbin/init"
depends_on:
- server1
Any suggestions ?
Building an image from your Dockerfile and running it with
docker run -p 2222:22 dschuldt/test
throws:
Could not load host key: /etc/ssh/ssh_host_rsa_key
Could not load host key: /etc/ssh/ssh_host_ecdsa_key
Could not load host key: /etc/ssh/ssh_host_ed25519_key
sshd: no hostkeys available -- exiting.
You can add this line to you dockerfile before the last CMD command to make it work (by the way, you have two CMD commands... the first one will be overwritten):
RUN /usr/bin/ssh-keygen -A
Just another small hint: Your Image is 739MB. Maybe you should rethink your use case ;-)
Have a nice evening, regards
dschuldt
Related
I'm trying to run a bash script after a Postgres container starts which 1) creates a new table within the Postgres DB, and 2) runs a copy command that dumps the contents of a csv file into the newly created table.
Currently, I'm specifying the execution of the script within my docker-compose.yml file using the "command" argument, but I find that it doesn't allow the Postgres container to succesfully start. I receive the following information from the log:
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
When I remove the "command" argument everything is fine. Here is what my docker-compose.yml files looks like now:
# docker-compose.yml
version: '3'
services:
web:
build: .
command: bash -c 'while !</dev/tcp/db/5432; do sleep 1; done; uvicorn app.main:app --host 0.0.0.0'
volumes:
- .:/app
expose: # new
- 8000
environment:
- DATABASE_URL=postgresql://fastapi_traefik:fastapi_traefik#db:5432/fastapi_traefik
depends_on:
- db
labels: # new
- "traefik.enable=true"
- "traefik.http.routers.fastapi.rule=Host(`fastapi.localhost`)"
db:
image: postgres:13-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
- "/Users/theComputerPerson/:/tmp"
expose:
- 5432
environment:
- POSTGRES_USER=fastapi_traefik
- POSTGRES_PASSWORD=fastapi_traefik
- POSTGRES_DB=fastapi_traefik
command: /bin/bash -c "/tmp/newtable.sh"
traefik: # new
image: traefik:v2.2
ports:
- 8008:80
- 8081:8080
volumes:
- "./traefik.dev.toml:/etc/traefik/traefik.toml"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
volumes:
postgres_data:
It may be worth noting that I'm trying to customize some of the aspects of this FastAPI project, and to turn your attention to the development files and not the production files. Please let me know if I can provide any additional information in the comments.
You are overriding the default container image startup command.
According to PostgreSQL official container image page, you can extend initialization adding your sh scripts (or even sql files) to /docker-entrypoint-initdb.d directory.
See https://hub.docker.com/_/postgres.
This approach has a caveat that this script could not be executed.
Another approach is to override default container image command adding yours in bash style: postgres; /bin/bash -c "/tmp/newtable.sh";
I am building a Flask application in Python. I'm using SQLAlchemy to connect to PostgreSQL.
In the flask application, I'm using this to connect SQLAlchemy to PostgreSQL
engine = create_engine('postgresql://postgres:[mypassword]#db:5432/employee-manager-db')
And this is my docker-compose.yml
version: '3.8'
services:
backend:
build:
context: .
dockerfile: Dockerfile
ports:
- 8000:8000
volumes:
- .:/app
links:
- db:db
depends_on:
- pgadmin
db:
image: postgres:14.5
restart: always
volumes:
- .dbdata:/var/lib/postgresql
hostname: postgres
environment:
POSTGRES_PASSWORD: [mypassword]
POSTGRES_DB: employee-manager-db
pgadmin:
image: 'dpage/pgadmin4'
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: [myemail]
PGADMIN_DEFAULT_PASSWORD: [mypassword]
ports:
- "5050:80"
depends_on:
- db
I can do "docker build -t employee-manager ." to build the image. However, when I do "docker run -p 5000:5000 employee-manager" to run the image, I get an error saying
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "db" to address: Try again
Does anybody know how to fix this? Thank you so much for your help
Your containers are on different networks and that is why they don't see each other.
When you run docker-compose up, docker-compose creates a separate network and puts all the services defined inside docker-compose.yml on that network. You can see that with docker network ls.
When you run a container with docker run, it is attached to the default bridge network, which is isolated from other networks.
There are several ways to fix this, but this one will serve you in many other scenarios:
Run docker container ls and identify the name or ID of the db container that was started with docker-compose
Then run your container with:
# ID_or_name from the previous point
docker run -p 5000:5000 --network container:<ID_or_name> employee-manager
This attached the new container to the same network as your database container.
Other ways include creating a network manually and defining that network as default in the docker-compose.yml. Then you can use docker run --network <network_name> ... to attach other containers to that network.
docker run doesn't read any of the information in the docker-compose.yml file, and it doesn't see things like the Docker network that Compose automatically creates.
In your case you already have the service fully-defined in the docker-compose.yml file, so you can use Compose commands to build and restart it
docker-compose build
docker-compose up -d # will delete and recreate changed containers
(If the name of the image is important to you – maybe you're pushing to a registry – you can specify image: alongside build:. links: are obsolete and you should remove them. I'd also avoid replacing the image's content with volumes:, since this misses any setup or modification that's done in the Dockerfile and it means you're running untested code if you ever deploy the image without the mount.)
First I have built an image using Dockerfile:
FROM openjdk:8-jdk-alpine
ARG JAR_FILE=target/*-SNAPSHOT.jar
ADD ${JAR_FILE} app.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/app.jar"]
as I have two docker-compose files one for production:
version: "3"
services:
app:
image: "demo:latest"
container_name: demo-production-api
restart: always
depends_on:
- "productiondb"
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://productiondb:5432/testdb
- SPRING_DATASOURCE_HIKARI_JDBC_URL=jdbc:postgresql://productiondb:5432/testdb
- SPRING_DATASOURCE_USER=tester
- SPRING_DATASOURCE_PASSWORD=test
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
ports:
- "8440:8443"
productiondb:
image: "postgres:latest"
container_name: productiondb
ports:
- "5430:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres-db-production:/usr/local/var/postgres
volumes:
postgres-db-production:
and one for develop:
version: "3"
services:
app:
image: "demo:latest"
container_name: demo-develop-api
restart: always
depends_on:
- "developdb"
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://developdb:5432/testdb
- SPRING_DATASOURCE_HIKARI_JDBC_URL=jdbc:postgresql://developdb:5432/testdb
- SPRING_DATASOURCE_USER=tester
- SPRING_DATASOURCE_PASSWORD=test
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
ports:
- "8441:8443"
developdb:
image: "postgres:latest"
container_name: developdb
ports:
- "5431:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres-db-develop:/usr/local/var/postgres
volumes:
postgres-db-develop:
I build both images using:
docker-compose -p demo-production-api -f docker-compose.yml up -d && docker-compose -p demo-develop-api -f docker-compose-develop.yml up -d
Now I was able to build both environments demo-develop-api and demo-production-api as well, the Spring Boot application from demo-develop-api docker image runs using the command:
docker run -it demo-develop-api
The application runs but I keep getting this error:
Caused by: java.net.UnknownHostException: productiondb
The above error happened after changing the database host in the application.properties file from localhost to productiondb first I was getting the following:
org.postgresql.util.PSQLException: Connection to localhost:5432
refused. Check that the hostname and port are correct and that the
postmaster is accepting TCP/IP connections.
Why this issue occurring or what is the cause?
How to solve this kind of issue?
As far as I see it, the issue might be that you have binded port 5430 and 5431 to 5432 and you might be having the port set to 5432 in your application.resources file. Your application should be trying to connect to the database by using either port 5430 or 5431 for production and development respectively. Please check and try this. So, make a port change in the application.resources file.
So after a long time of debugging and trials, hopefully, this is is going to save people hours, it turned out that actually, the Spring Boot application inside the container was restarting runs and crashes without any errors, which made me more confused why it is not listening or opening a port. I even doubt it that it could be a firewall or something. So basically I just tried to get a shell from the container by doing:
docker exec -it <container id or image> sh
Note: Since I am using the image openjdk:8-jdk-alpine don't do below you will not get a shell:
docker exec -it <container id or image> bash
Then I tried to get a list of open ports by doing:
netstat -tulpn | grep ":8443"
The port 8443 was not listed, I thought it could be a problem with the java program not being running, tried to execute the spring boot which executed but without any errors and the shell itself was exiting which made me more confused.
Until I have found out that container was restartig because of Spring Boot was crashing. So I enabled verbose mode by adding the below properties to application.properties then rebuild the image again:
logging.level.org.springframework.web=DEBUG
logging.level.org.hibernate=DEBUG
So I retried the last above steps where I get a shell and execute the app.jar and it turned out that the database testdb did not exist.
UPDATE: So to sum up here how I modifed my project, I created two Spring Boot Profiles for my case one for develop application-develop.properties and one for production application-production.properties:
So inside the application-develop.properties I have it mapped to a develop postgres container host and port:
spring.datasource.url=jdbc:postgresql://developdb:5432/testdb
spring.datasource.hikari.jdbc-url=jdbc:postgresql://developdb:5432/testdb
spring.datasource.username=tester
spring.jpa.generate-ddl=true
spring.datasource.password=test
spring.jpa.database-platform=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL9Dialect
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
server.port=8443
And for application-production.properties:
spring.datasource.url=jdbc:postgresql://productiondb:5432/testdb
spring.datasource.hikari.jdbc-url=jdbc:postgresql://productiondb:5432/testdb
spring.datasource.username=tester
spring.jpa.generate-ddl=true
spring.datasource.password=test
spring.jpa.database-platform=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL9Dialect
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
server.port=8443
And in the docker-compose file for develop I just define the Spring Boot profile environment variable to:
environment:
- SPRING_PROFILES_ACTIVE=develop
And for production docker-compose file I define it as below:
environment:
- SPRING_PROFILES_ACTIVE=production
I have a scrapy application which I'm trying to containerized it. Basically, this is my docker-compose.yml file:
version: '3'
services:
scrapper:
container_name: scrapper
build: .
ports:
- 80:80
depends_on:
- db
links:
- db
db:
volumes:
- ./scrapper/sql:/docker-entrypoint-initdb.d
image: postgres
container_name: postgres
restart: always
ports:
- 5432:5432
And this is my Dockerfile:
FROM python:3
WORKDIR /usr/app
COPY requirements.txt .
RUN pip3 install -r requirements.txt
COPY . .
But when I try to execute my application using the following command: docker run -it scrapper_scrapper scrapy crawl angeloni, I'm receiving this message:
File "/usr/local/lib/python3.7/site-packages/scrapy/crawler.py", line 88, in crawl
yield self.engine.open_spider(self.spider, start_requests)
psycopg2.OperationalError: could not translate host name "db" to address: Name or service not known
Why this is happening? When I execute docker-compose ps command, it shows:
Name Command State Ports
--------------------------------------------------------------------------
postgres docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp
scrapper python3 Exit 0
When running docker-compose up to start db, that container will run under its network that is also created by docker compose. As such, running docker run ... will not be able to connect to that instance, since it is not running on the same network. But you can specify it with:
docker run --network $network_name
To get the docker networks available, you can run:
docker networks ls
I think you have to explicitly define a user network and put your containers on it:
https://docs.docker.com/network/bridge/
Under the section:
User-defined bridges provide automatic DNS resolution between containers.
I m new to docker
so i followed up a tutorial here , part 6,7 and 8 in order to use and learn docker in a project.
The problem is that when i use docker-compose.yml to build images on my laptop,
my stack_server can connect to mongo.
But if i build images from docker hub and i pull and run them seperately on my laptop,my stack_server CAN't connect to mongo.
here is my docker-compose.yml :
client:
build: ./client
restart: always
ports:
- "80:80"
links:
- server
mongo:
image: mongo
command: --smallfiles
restart: always
ports:
- "27017:27017"
server:
build: ./server
restart: always
ports:
- "8080:8080"
links:
- mongo
However, my stack_client can connect to the stack_server.
and my commands to run my image (my images are public)
docker-run -i -t -p 27017:27017 mongo
docker-run -i -t -p 80:80 mik3fly4steri5k/stack_client
docker-run -i -t -p 8080:8080 mik3fly4steri5k/stack_server
and my error log
bryan#debian-dev7:~$ sudo docker run -i -t -p 8080:8080 mik3fly4steri5k/stack_server
[sudo] password for bryan:
Express server listening on 8080, in development mode
connection error: { Error: connect ECONNREFUSED 127.0.0.1:27017
at Object._errnoException (util.js:1021:11)
at _exceptionWithHostPort (util.js:1043:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1175:14)
name: 'MongoError',
message: 'connect ECONNREFUSED 127.0.0.1:27017' }
here is my stack_server dockerfile
FROM node:latest
# Set in what directory commands will run
WORKDIR /home/app
# Put all our code inside that directory that lives in the container
ADD . /home/app
# Make sure NPM is not cached, remove everything first
RUN rm -rf /home/app/node_modules/npm \
&& rm -rf /home/app/node_modules
# Install dependencies
RUN npm install
# Tell Docker we are going to use this port
EXPOSE 8080
# The command to run our app when the container is run
CMD ["node", "app.js"]
First Solution but deprecated
docker-run -i -t -p 27017:27017 --name:mongo mongo
docker-run -i -t -p 80:80 mik3fly4steri5k/stack_client
docker-run -i -t -p 8080:8080 --link mongo:mongo mik3fly4steri5k/stack_server
I had to put a name on my mongo and to link it with the parameter --link when i run my stack_server
docker documentation
Warning: The --link flag is a deprecated legacy feature of Docker.
It may eventually be removed.
Unless you absolutely need to continue using it,
we recommend that you use user-defined networks to facilitate communication
between two containers instead of using --link.
One feature that user-defined networks do not support that you can do with
--link is sharing environmental variables between containers.
However, you can use other mechanisms such as volumes to share environment
variables between containers in a more controlled way.
Run your stack_server with below command
sudo docker run -i -t -p 8080:8080 --link mongo:mongo mik3fly4steri5k/stack_server
Note: --link flag is a deprecated
Your container is running. But is it healthy? I recommend that you implement Healthcheck
It’s actually would look pretty much the same:
(Docker-compose YAML)
healthcheck:
test: curl -sS http://127.0.0.1:8080 || echo 1
interval: 5s
timeout: 10s
retries: 3
Docker health checks is a cute little feature that allows attaching shell command to container and use it for checking if container’s content is alive enough.
I hope it can help you.
more info https://docs.docker.com/compose/compose-file/compose-file-v2/#healthcheck
To start MongoDB on a desire port you would have to do the followings:
client:
build: ./client
restart: always
ports:
- "80:80"
links:
- server
mongo:
image: mongo
command: mongod --port 27017
restart: always
ports:
- "27017:27017"
server:
build: ./server
restart: always
ports:
- "8080:8080"
links:
- mongo
Lets look at error
connection error: { Error: connect ECONNREFUSED 127.0.0.1:27017
Server trying to connect to localhost instead of mongo. You need to configure server to connect mongo to mongo:27017
mongo is alias created by docker for linking containers to each other.