In order to integrate Superset with my application properly, I need to enable HTTPS. During development I'm running it with docker-compose. I couldn't find any useful information on how to do that. The version I'm running is v1.1.0
I'd be very glad if you could help me. Thank you
Clone superset repo
git clone https://github.com/apache/superset.git
add nginx-proxy as service in docker-compose.yaml
nginx-proxy:
image: jwilder/nginx-proxy
ports:
- "443:443"
volumes:
- .certs:/etc/nginx/certs
- /var/run/docker.sock:/tmp/docker.sock:ro
environment:
DEFAULT_HOST: superset.localhost
add VIRTUAL_HOST and WEB_PORTS for superset service in docker-compose.yaml
environment:
CYPRESS_CONFIG: "${CYPRESS_CONFIG}"
VIRTUAL_HOST: superset.localhost
WEB_PORTS: 8088
Let's create self signed certificate for HTTPS.
mkdir .certs && cd .certs
wget https://gist.githubusercontent.com/OnnoGabriel/f717192ed92bf55725337358f4af5ab2/raw/9b669462299c9981bd7864901f09fc2885d9e780/create_certificates.sh
sudo chmod 700 ./create_certificates.sh
sudo ./create_certificates.sh
enter-domain name --> superset.localhost
To make a browser familiar with our root CA, we have to import them into browsers.
Firefox --> https://support.securly.com/hc/en-us/articles/360008547993-How-to-Install-Securly-s-SSL-Certificate-in-Firefox-on-Windows
Chrome --> https://support.securly.com/hc/en-us/articles/206081828-How-to-manually-install-the-Securly-SSL-certificate-in-Chrome
take rootCA.crt file and add them into your browsers based on above instruction.
now start your superset docker instance by
docker-compose -f docker-compose.yml up -d
Now you can access superset by https://superset.localhost
For more details checkout this article
https://betterprogramming.pub/docker-powered-web-development-utilizing-https-and-local-domain-names-a57f129e1c4d
Related
I would really appreciate some help with the current issue I am experiencing.
Context:
I have been upgrading my instance of keycloak from 16.x to 18.x.
After many hours of research, I have been defeated by this one issue.
Issue:
When I go to the site URL for this example https://thing.com/ I am greeted with the following "Resource not found", instead of the keycloak welcome page.
In my chrome network monitoring it will show the following:
Error with network monitor
Infra:
Keycloak lives on its machine. The URL reaches keycloak through a Caddy Service as a reverse proxy.
Relative scripts:
Docker-compose
version: "3.1"
services:
keycloak:
image: quay.io/keycloak/keycloak:18.0.2
environment:
JAVA_OPTS: "-XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0 -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=\"org.jboss.byteman\" -Djava.awt.headless=true"
KC_HOSTNAME_PORT: 8080
KC_HOSTNAME: ${KC_HOME}
KC_PROXY: edge
KC_DB_URL: 'jdbc:postgresql://${KEYCLOAK_DB_ADDR}/${KEYCLOAK_DB_DATABASE}?sslmode=require'
KC_DB: postgres
KC_DB_USERNAME: ${KEYCLOAK_DB_USER}
KC_DB_PASSWORD: ${KEYCLOAK_DB_PASSWORD}
KC_HTTP_RELATIVE_PATH: /auth
KC_HOSTNAME_STRICT_HTTPS: 'false'
command: start --auto-build
ports:
- 8080:8080
- 8443:8443
volumes:
- backup:/var/backup
healthcheck:
test: curl -I http://127.0.0.1:8080/
volumes:
backup:
NOTE: If I remove this KC_HTTP_RELATIVE_PATH: /auth it will behave as intended. However, I would prefer I do not remove this aspect of the service as it is tied to that relative path for a lot of the services using keycloak.
I can replicate this with a local docker image built using the same environment variables.
Does anyone perhaps know some secret ninja moves I could do to get it to direct to the welcome page?
Automatic redirect from / to KC_HTTP_RELATIVE_PATH is not supported in Keycloak 18 (see https://github.com/keycloak/keycloak/discussions/10274).
You have to add the redirect in the reverse proxy, in Caddy there is redir.
I have a docker-compose.yml with two services, a custom image (the service called code) and a Postgres server. Below I attach the Dockerfile to built the image called app of the first service and next the docker-compose.yml:
# Dockerfile of custom image
FROM ubuntu:latest
RUN apt-get update \
&& apt-get install -y python3-pip python3-dev \
&& cd /usr/local/bin \
&& ln -s /usr/bin/python3 python \
&& pip3 install --upgrade pip
WORKDIR /usr/app
COPY ./* ${PWD}/
ADD ./requirements.txt ./
RUN pip install -r requirements.txt
ADD ./ ./
# docker-compose.yml
version: '3.2'
services:
code:
image: app:latest
ports:
- 5001:5001
networks:
- docker-elk_elk
postgres:
image: postgres:9.5-alpine
environment:
POSTGRES_USER: postgres # define credentials
POSTGRES_PASSWORD: postgres # define credentials
POSTGRES_DB: postgres # define database
ports:
- 5432:5432 # Postgres port
networks:
- docker-elk_elk
networks:
docker-elk_elk:
external: true
Also here docker-elk_elk points to another network where another docker-compose stack runs and I want the above docker-compose to join too. However when I run docker-compose run code bash and obtain a shell in code service, a curl https://postgres:5432 gives the following message : curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to postgres:5432. I've tried also curl http://postgres:5432 which returned curl: (52) Empty reply from server. Furthermore, the docker-elk_elk network (clearly created by elasticsearch-logtash-kibaba stack) when doing docker network ls gives
NETWORK ID NAME DRIVER SCOPE
8a54fe394fe8 docker-elk_elk bridge local
I'm really lost and confused, can someone help me out? If there is any piece of info that might be necessary or helpful and wasn't included above let me know please.
I forgot to mention that app is just a simple python application (not a web app or other python sophisticated libraries).
P.S. Something that perhaps I should have mentioned above. What I want to do, is using the ubuntu container with the app inside to query (and send data) both to the postgres and Elasticsearch (which is in the other docker-compose stack) db.
I have built a RESTful API web service using Flask framework, Redis as main database, MongoDB as a backup store and Celery as task queue to store data into MongoDB in background
Then I dockerize my application using docker-compose. Here is my docker-compose.yml:
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/app
redis:
image: "redis:alpine"
ports:
- "6379:6379"
mongo:
image: "mongo:3.6.5"
ports:
- "27017:27017"
environment:
MONGO_INITDB_DATABASE: syncapp
Here is my Dockerfile:
# base image
FROM python:3.5-alpine
MAINTAINER xhoix <145giakhang#gmail.com>
# copy just the requirements.txt first to leverage Docker cache
# install all dependencies for Python app
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
# install dependencies in requirements.txt
RUN pip install -r requirements.txt
# copy all content to work directory /app
COPY . /app
# specify the port number the container should expose
EXPOSE 5000
# run the application
CMD ["python", "/app/app.py"]
After run command docker-compose up, the app server, Redis and Mongo server just run well. But when I use Postman or curl to call the API, for example http://127.0.0.1:5000/sync/api/v1.0/users, which should return JSON format of all users, but the result is Could not get any response: There was an error connecting to http://127.0.0.1:5000/sync/api/v1.0/users.
I have no idea why this happens.
Thanks for any help and suggestion!
I found the cause of the issue:
After an hour debug, it turns out that I only need to change the app host to 0.0.0.0. Maybe when mapping port, docker default will be 0.0.0.0, since when I run command docker-compose ps, the PORTS column of each container has format 0.0.0.0:<port> -> <port>. I don't know this is the cause of the issue or not, but I did it and the problem is solved
If operating system Linux then use :
ifconfig -a
If operating system Windows then use :
ipconfig /all
Then check the interface like docker or something with virtualization, and use the ipv4 or inet
Or Just use the docker command:
docker network inspect bridge
Then use the gateway ip on IPAM
I've been playing with Docker for the past week and think the container idea is very useful, but despite reading everything I can for the past 3 days I can't get the volume mapping to work
get docker-compose to use my existing volume.
Docker Version: 18.03.1-ce
docker-compose version 1.21.1, build 7641a569
I created a volume using the following via a Dockerfile
# Reference SQL image
FROM microsoft/mssql-server-windows-developer
# Create directory within SQL container for database files mapped to the volume
VOLUME sqldata:c:/MSSQL
and here it shows:
C:\ProgramData\Docker\volumes>docker volume ls
local sqldata
Now I've tried probably 60+ different "solutions" based on StackOverflow and Docker forums, but none of them work. (Note despite the names below with Azure I am simply trying to get this to run locally, Azure is next hurdle)
Docker-compose.yaml:
version: '3.4'
services:
ws:
image: wsManager
container_name: azure-wcf
ports:
- "80"
depends_on:
- db
db:
image: dbimage:latest
container_name: azure-db
volumes:
- \sqldata:/mssql
# - type: volume
# source: sqldata
# target: /mssql
ports:
- "1433"
I've added a volumes section but it does not help,
volumes:
sqldata:
external:
name: sqldata
changed the - \sqldata:/mssql
to every possible slash .. . ~ whatever. Moved the file to yaml file
to C:\ProgramData\Docker\volumes - basically any suggestion that showed in my search results. The dbImage is a SQL Server image that I need to persist the data from but am wondering what the magic is as nothing I've tried works. Any help is GREATLY appreciated.
I'm running on Windows 10 Pro build 1803.
Why does this have to be so hard?
Than you to whomever knows how to make this actually work.
The solution is to reference the true path on Windows using the volumes: option as below:
sqldb:
image: sqlimage
container_name: azure-db
volumes:
- "C:\\ProgramData\\Docker\\volumes\\sqldata:c:\\mssql"
To persist the data I used the following:
environment:
- "sa_password=ddsql2017##"
- "ACCEPT_EULA=Y"
- 'attach_dbs= {"dbName":"MyDb","dbFiles":"C:\\MSSQL\\MyDb.mdf","C:\\MSSQL\\MyDb.ldf"]}]'
Hope this helps someone else as many of the examples I found searching both on SO and elsewhere did not work for me, and in the Docker forums there are a lot of posts saying mounting volumes not work for Windows.
For those who are using Ubunto WSL:
sudo mkdir /c
sudo mount --bind /mnt/c /c
navigate to your project file use new path ( /c/your-project-path and not /mnt/c/your-project-path)
edit your docker-compose.yml and use relative path for volume : ( like ./src instead of c/your-project-path/src)
docker-compose up
I was struggling with a similar problem when trying to mount a volume to a specific path of my Windows machine: basically it didn't work so every time I restarted my Docker instance I lose all my DB data.
I finally found out that it is because Docker for Windows by default cannot interpret Windows path so the flag COMPOSE_CONVERT_WINDOWS_PATHS has to be activated. To do so:
Run the command "set COMPOSE_CONVERT_WINDOWS_PATHS=1"
Restart Docker
Go to Settings > Shared Drives > Reset credentials and then select drive and then apply
From the command line, kill the containers (docker container rm -f )
Re-run the containers
Hope it helps
If your windows account credentials has been changed, you also have to reset credentials for shared drives. (Settings > Shared Drives > Reset credentials)
In my case, the password was changed by my company security policy.
Are you sure you really need to map to a certain host directory? If not, my solution is to create a volume beforehand and use it in docker-compose.yaml. I use the same scripts for both windows and linux. That is the beauty of docker.
Here is what I did to start both postgres and mysql:
create_db.sh (you can run it in git bash or similiar environment in windows):
docker volume create --name postgres-data -d local
docker volume create --name mysql-data -d local
docker-compose up -d
docker-compose.yaml:
version: '3'
services:
postgres:
image: postgres:latest
environment:
POSTGRES_DB: datasource
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
volumes:
- postgres-data:/var/lib/postgresql/data
mysql:
image: mysql:latest
environment:
MYSQL_DATABASE: 'train'
MYSQL_USER: 'mysql'
MYSQL_PASSWORD: 'mysql'
MYSQL_ROOT_PASSWORD: 'mysql'
ports:
- 3306:3306
volumes:
- mysql-data:/var/lib/mysql
volumes:
postgres-data:
external: true
mysql-data:
external: true
By default it looks that after installing Docker on Windows, sharing of drivers is disabled - so you won't be able to use volumes(that are stored on disks)
Enabling such sharing, through: Docker in tray - right click - Settings, helped to me, volumes started working fine.
Docker on Windows is having strange behavior as Windows has limitations with credentials and also with the virtual machine that Docker is using(Hyper-V , VirtualBox - depending on your Docker version and setup).
Basically, you are correct to map a folder in
volumes:
section in your service:
The path is
version: '3.4'
services:
db:
image: dbimage:latest
container_name: azure-db
volumes:
- c:/Temp/sqldata:/mssql
Important is that you do not need to explicitly create volume in volumes section, but the docker-compose up will create it(the same is for docker run).
Strange thing is that it will never show up in
docker volume ls
but it will be usable with the same files inside windows directory and inside container path /mssql
You can test it with:
docker run --rm -v c:/Temp/sqldata:/data alpine ls /data
or
docker run --rm -v c:/Temp:/data alpine ls /data
If it Disappear, probably it lost the credentials and Reset it via Docker->Settings->Shared Drives->Reset credentials.
I hope it was clear and covered all the aspects for you.
Launch Docker from your windows taskbar
Click on Settings icon on top
Click Resources
Click File Sharing
Click on (+) sign and add path of local folder in which you want to map the container volume.
It worked for me.
this is a circleci question I guess.
I am quite happy with circleci but now I ran into a problem and I don't know what I'm doing wrong.
Maybe this is something very easy, but I don't see the it.
In short
I can't make containers talk to each other on circleci.
Problem
Basically what I wanted to do is start a server container and a client container, and then let them talk to each other.
I created a minimal example here: https://github.com/mRcSchwering/circleci-integration-test
The README.md basically explains the desired outcome.
I have a .circleci/config.yml like this:
version: 2
jobs:
build:
docker:
- image: docker:18.03.0-ce-git
steps:
- checkout
- setup_remote_docker
- run:
name: Install docker-compose
command: |
apk --update add py2-pip
/usr/bin/pip2 install docker-compose
docker-compose --version
- run:
name: Start Container
command: |
docker-compose up -d
docker-compose ps
- run:
name: Let client talk to server
command: |
docker-compose run client psql -h server -p 5432 -U postgres -c "\l"
In a docker container, docker-compose is installed, which is then used to start a server and a client (postgres here). In the last step I am telling the client to query the server. However, it cannot find the server:
#!/bin/sh -eo pipefail
docker-compose run client psql -h server -p 5432 -U postgres -c "\l"
Starting project_server_1 ...
^#^#psql: could not connect to server: Connection refused
Is the server running on host "server" (172.18.0.2) and accepting
TCP/IP connections on port 5432?
Exited with code 2
Files
The docker-compose.yml looks like this
version: '2'
services:
server:
image: postgres:9.5.12-alpine
networks:
- internal
expose:
- '5432'
client:
build:
context: .
networks:
- internal
depends_on:
- server
networks:
internal:
driver: bridge
where the client is built from a dockerfile like this
FROM alpine:3.7
RUN apk --no-cache add postgresql-client && rm -rf /var/cache/apk/*
Note
If I repeat everything on my Linux (also with docker-in-docker) it works.
But I guess some things work completely different on circleci.
I found some people mentioning that on circleci networking and bind mounts can be tricky but I didn't find anything that can help me.
There is this doc but I thought I am doing this already.
Then there is this project where someone seems to do the same thing on circleci successfully.
But I cannot figure out what's different there...
Anyway I would really appreciate your help. So far I have given up on this.
Best
Marc
Ok, in the meanwhile I (no actually it was halfer from the circleci forum) noticed that docker-compose run client psql -h server -p 5432 -U postgres -c "\l" was run before the server was up and running. A simple sleep 5 after docker-compose up -d fixes the problem.