I have been trying for several days to run gitlab via docker-composing through Traefik. I found several topics about it but none of them really solved my problem. So I would like to open a topic today.
I have helped myself a lot with the following example: https://github.com/ambroisemaupate/docker-server-env/blob/master/compose/example-gitlab-traefik/docker-compose.yml
version: '3'
services:
traefik:
container_name: traefik
image: traefik:1.7.3 # The official Traefik docker image
restart: always
command: --api --docker # Enables the web UI and tells Traefik to listen to docker
ports:
- "80:80" # The HTTP port
- "443:443" # The HTTPS port
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
- ./conf/traefik.toml:/traefik.toml
- ./conf/acme.json:/acme.json
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik_network"
- "traefik.port=80"
- "traefik.entryPoint=https"
- "traefik.backend=traefik"
- "traefik.frontend.rule=Host:traefik.domain.com"
networks:
- traefik_network
gitlab:
container_name: gitlab
image: 'gitlab/gitlab-ce:latest'
restart: always
hostname: 'gitlab.domain.com'
labels:
- "traefik.docker.network=traefik_network"
- "traefik.enable=true"
- "traefik.port=80"
- "traefik.frontend.rule=Host:https://gitlab.domain.com"
- "traefik.frontend.entryPoints=https"
healthcheck:
disable: true
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.giftdigging.com'
nginx['redirect_http_to_https'] = false
nginx['listen_port'] = 80
nginx['listen_https'] = false
nginx['proxy_set_headers'] = {
'X-Forwarded-Proto' => 'https',
'X-Forwarded-Ssl' => 'on'
}
nginx['client_max_body_size'] = '2G'
unicorn['worker_timeout'] = 60
unicorn['worker_processes'] = 2
sidekiq['concurrency'] = 15
postgresql['shared_buffers'] = "512MB"
gitlab_rails['artifacts_enabled'] = true
gitlab_rails['artifacts_path'] = "/var/opt/gitlab/gitlab-artifacts"
gitlab_rails['lfs_enabled'] = true
gitlab_rails['backup_keep_time'] = 172600
gitlab_ci['backup_keep_time'] = 172600
ports:
- '22:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
networks:
- traefik_network
networks:
traefik_network:
external: true
internal_network:
external: false
and my traefik.toml
debug = false
logLevel = "INFO"
defaultEntryPoints = ["https", "http"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[retry]
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "domain.com"
watch = true
exposedByDefault = false
[acme]
email = "contact#monsite.co"
storage = "acme.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"
I get a safety warning and a certificate self-signed by Traefik.
And, if I accept the risk despite the warning of my browser, I get a 404 page not found.
This is a syntax error in the next line:
- "traefik.frontend.rule=Host:https://gitlab.domain.com"
The protocol should not be specified.
The line must correspond to:
- "traefik.frontend.rule=Host:gitlab.domain.com"
Related
As per the official documentation (https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-google-cloud-gke), the following works as expected:
helm install vault hashicorp/vault \
--set='server.ha.enabled=true' \
--set='server.ha.raft.enabled=true'
I can then run:
kubectl exec vault-0 -- vault status
And this works perfectly fine.
However, I've noticed that when if I don't have raft enabled, I get the dial tcp 127.0.0.1:8200: connect: connection refused" error message:
helm install vault hashicorp/vault \
--set='server.ha.enabled=true'
I'm trying to work out why my Vault deployment is giving me the same issue.
I'm trying to deploy Vault into GKE with auto-unseal keys and a Google Cloud Storage backend configured.
My values.yaml file contains:
global:
enabled: true
tlsDisable: false
injector:
enabled: true
replicas: 1
port: 8080
leaderElector:
enabled: true
image:
repository: "hashicorp/vault-k8s"
tag: "latest"
pullPolicy: IfNotPresent
agentImage:
repository: "hashicorp/vault"
tag: "latest"
authPath: "auth/kubernetes"
webhook:
failurePolicy: Ignore
matchPolicy: Exact
objectSelector: |
matchExpressions:
- key: app.kubernetes.io/name
operator: NotIn
values:
- {{ template "vault.name" . }}-agent-injector
certs:
secretName: vault-lab.company.com-cert
certName: tls.crt
keyName: tls.key
server:
enabled: true
image:
repository: "hashicorp/vault"
tag: "latest"
pullPolicy: IfNotPresent
extraEnvironmentVars:
GOOGLE_APPLICATION_CREDENTIALS: /vault/userconfig/vault-gcs/service-account.json
GOOGLE_REGION: europe-west2
GOOGLE_PROJECT: sandbox-vault-lab
volumes:
- name: vault-gcs
secret:
secretName: vault-gcs
- name: vault-lab-cert
secret:
secretName: vault-lab.company.com-cert
volumeMounts:
- name: vault-gcs
mountPath: /vault/userconfig/vault-gcs
readOnly: true
- name: vault-lab-cert
mountPath: /etc/tls
readOnly: true
service:
enabled: true
type: NodePort
externalTrafficPolicy: Cluster
port: 8200
targetPort: 8200
annotations:
cloud.google.com/app-protocols: '{"http":"HTTPS"}'
beta.cloud.google.com/backend-config: '{"ports": {"http":"config-default"}}'
ha:
enabled: true
replicas: 3
config: |
listener "tcp" {
tls_disable = 0
tls_min_version = "tls12"
address = "[::]:8200"
cluster_address = "[::]:8201"
}
storage "gcs" {
bucket = "vault-lab-bucket"
ha_enabled = "true"
}
service_registration "kubernetes" {}
# Example configuration for using auto-unseal, using Google Cloud KMS. The
# GKMS keys must already exist, and the cluster must have a service account
# that is authorized to access GCP KMS.
seal "gcpckms" {
project = "sandbox-vault-lab"
region = "global"
key_ring = "vault-helm-unseal-kr"
crypto_key = "vault-helm-unseal-key"
}
Something here must be misconfigured, but what, I'm unsure.
Any help would be appreciated.
EDIT:
Even after configuring Raft, I still encounter the same issue:
raft:
enabled: true
setNodeId: false
config: |
ui = false
listener "tcp" {
# tls_disable = 0
address = "[::]:8200"
cluster_address = "[::]:8201"
tls_cert_file = "/etc/tls/tls.crt"
tls_key_file = "/etc/tls/tls.key"
}
#storage "raft" {
# path = "/vault/data"
#}
storage "gcs" {
bucket = "vault-lab-bucket"
ha_enabled = "true"
}
service_registration "kubernetes" {}
I am receiving the following error when trying to call a service using Dapr SDK.
System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:3500)
---> System.Net.Sockets.SocketException (111): Connection refused
Here is my docker-compose settings of the service I am trying to call:
quest-service:
image: ${DOCKER_REGISTRY-gamification}/quest-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
build:
context: .
dockerfile: Services/LW.Gamification.QuestService/Dockerfile
ports:
- "5110:80"
- "50010:50001"
quest-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Quest-Service",
"-app-port", "80",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- quest-service
network_mode: "service:quest-service"
And the settings for the caller:
player-service:
image: ${DOCKER_REGISTRY-gamification}/player-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
build:
context: .
dockerfile: Services/LW.Gamificaiton.PlayerService/Dockerfile
ports:
- "5109:80"
- "50009:50001"
player-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Player-Service",
"-app-port", "80",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- player-service
network_mode: "service:player-service"
And here is the code that is failing to work:
// demo service to service call
var httpClient = DaprClient.CreateInvokeHttpClient("Quest-Service");
var requestUri = $"api/v1/Quest";
var result = await httpClient.GetFromJsonAsync<IEnumerable<string>>(requestUri);
Note: Messaging is working fine. :-)
I am new to Dapr so I must be doing something silly wrong, maybe something to do with ports.. I just don't know!
From following this question :Dapr Client Docker Compose Issue
I managed to get this partly working using the following docker-compose config:
services:
placement:
image: "daprio/dapr"
command: ["./placement", "-port", "50000", "-log-level", "debug"]
ports:
- "50000:50000"
quest-service:
image: ${DOCKER_REGISTRY-gamification}/quest-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
- DAPR_GRPC_PORT=50010
build:
context: .
dockerfile: Services/LW.Gamification.QuestService/Dockerfile
ports:
- "5110:80"
- "50010:50010"
depends_on:
- placement
- rabbitmq
- redis
- seq
- zipkin
quest-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Quest-Service",
"-app-port", "80",
"-placement-host-address", "placement:50000",
"-dapr-grpc-port", "50010",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- quest-service
network_mode: "service:quest-service"
generatetraffic:
image: ${DOCKER_REGISTRY-gamification}/generatetraffic:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
- DAPR_GRPC_PORT=50017
build:
context: .
dockerfile: Services/LW.Gamification.GenerateTraffic/Dockerfile
ports:
- "5117:80"
- "50017:50017"
depends_on:
- placement
- rabbitmq
- redis
- seq
- zipkin
generatetraffic-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Generate-Traffic",
"-app-port", "80",
"-placement-host-address", "placement:50000",
"-dapr-grpc-port", "50017",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- generatetraffic
network_mode: "service:generatetraffic"
However I still have issues with some of the documented APIs not working!.
var httpClient = DaprClient.CreateInvokeHttpClient("Quest-Service");
var requestUri = $"api/v1/Quest";
var result = await httpClient.GetAsync(requestUri);
Still fails?
I try to set up my own mailserver, Mailcow was recommended.
DNS-provider:
Cloudflare with
CNAME mail.examle.com => examle.com, proxied
Because it is proxies, I cannot use normal ports like mentioned in the docs. Therefore I have to setup some forwarding...
Router:
Fritzbox with port forwadring
2052 => 25
2053 => 465
8080 => 587
2082 => 143
2083 => 993
2086 => 110
2087 => 995
8880 => 4190
Docker:
I use jwilders reverse proxy and it's LE-companion, which works well with everything else I have hosted so far.
${DOCKERDIR}/docker-compose-js.yml
version: '3'
services:
proxy:
build: ./reverse_proxy
container_name: proxy
restart: always
ports:
- 80:80
- 443:443
volumes:
- ${DOCKERDIR}/reverse_proxy/certs:/etc/nginx/certs:ro
- ${DOCKERDIR}/reverse_proxy/vhost.d:/etc/nginx/vhost.d
- ${DOCKERDIR}/reverse_proxy/html:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
environment:
- PUID=33
- PGID=33
labels:
com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: ""
networks:
- proxy-tier
depends_on:
- le
le:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: le
volumes:
- ${DOCKERDIR}/reverse_proxy/certs:/etc/nginx/certs:rw
- ${DOCKERDIR}/reverse_proxy/vhost.d:/etc/nginx/vhost.d
- ${DOCKERDIR}/reverse_proxy/html:/usr/share/nginx/html
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- PUID=33
- PGID=33
- DEFAULT_EMAIL=*****
- NGINX_PROXY_CONTAINER=proxy
networks:
- proxy-tier
networks:
proxy-tier:
Then there is a (slightly) modified file for mailcow, just mentioning the changes
%{DOCKERDIR}/mailcow/docker-compose.yml
nginx-mailcow:
...
# ports:
# - "${HTTPS_BIND:-0.0.0.0}:${HTTPS_PORT:-443}:${HTTPS_PORT:-443}"
# - "${HTTP_BIND:-0.0.0.0}:${HTTP_PORT:-80}:${HTTP_PORT:-80}"
...
There seems to be no way to remove those ports from it's original docker-compose.yml despite it not being recommended.
For all other changes I got
${DOCKERDIR}/mailcow/docker-compose-override.yml
version: '2.1'
services:
nginx-mailcow:
networks:
proxy-tier:
environment:
- VIRTUAL_HOST=${MAILCOW_HOSTNAME},${ADDITIONAL_SAN}
- VIRTUAL_PORT=8080
- VIRTUAL_PROTO=http
- LETSENCRYPT_HOST=${MAILCOW_HOSTNAME},${ADDITIONAL_SAN}
volumes:
- ${DOCKERDIR}/reverse_proxy/certs/${MAILCOW_HOSTNAME}:/etc/ssl/mail/
- ${DOCKERDIR}/reverse_proxy/certs/dhparam.pem:/etc/ssl/mail/dhparams.pem:ro
ports:
dovecot-mailcow:
volumes:
- ${DOCKERDIR}/reverse_proxy/certs/${MAILCOW_HOSTNAME}:/etc/ssl/mail/
- ${DOCKERDIR}/reverse_proxy/certs/dhparam.pem:/etc/ssl/mail/dhparams.pem:ro
postfix-mailcow:
volumes:
- ${DOCKERDIR}/reverse_proxy/certs/${MAILCOW_HOSTNAME}:/etc/ssl/mail/
- ${DOCKERDIR}/reverse_proxy/certs/dhparam.pem:/etc/ssl/mail/dhparams.pem:ro
networks:
proxy-tier:
And finally the mailcow.conf (changes only)
${DOCKERDIR}/mailcow/mailcow.conf
MAILCOW_HOSTNAME=mail.example.com
HTTP_PORT=8080
#HTTP_BIND=0.0.0.0
HTTP_BIND=proxy
HTTPS_PORT=8443
#HTTPS_BIND=0.0.0.0
HTTPS_BIND=proxy
SKIP_LETS_ENCRYPT=y
When I try to connect to mail.example.com I get Error 526 Invalid SSL certificate.
Could someone pls show me where my config is wrong and how to change it so I get mailcow working?
I'm trying to use Keyrock to offer Single Sign-on on different platforms. Specifically, I want to offer that service in Grafana. I've seen the configuration to be changed in Grafana and my docker-compose is like this:
version: "3.1"
services:
grafana:
image: grafana/grafana:5.1.0
ports:
- 3000:3000
networks:
default:
ipv4_address: 172.18.1.4
environment:
- GF_AUTH_GENERIC_OAUTH_CLIENT_ID=90be8de5-69dc-4b9a-9cc3-962cca534410
- GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=9e98964b-5043-4086-9657-51f1d8c11fe0
- GF_AUTH_GENERIC_OAUTH_ENABLED=true
- GF_AUTH_GENERIC_OAUTH_AUTH_URL=http://172.18.1.5:3005/oauth2/authorize
- GF_AUTH_GENERIC_OAUTH_TOKEN_URL=http://172.18.1.5:3005/oauth2/token
- GF_AUTH_GENERIC_OAUTH_API_URL=http://172.18.1.5:3005/v1/users
- GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP = true
- GF_Server_DOMAIN=172.18.1.4
- GF_Server_ROOT_URL=http://172.18.1.4:3000
keyrock:
image: fiware/idm:7.5.1
container_name: fiware-keyrock
hostname: keyrock
networks:
default:
ipv4_address: 172.18.1.5
depends_on:
- mysql-db
ports:
- "3005:3005"
- "3443:3443"
environment:
- DEBUG=idm:*
- DATABASE_HOST=mysql-db
- IDM_DB_PASS_FILE=/run/secrets/my_secret_data
- IDM_DB_USER=root
- IDM_HOST=http://localhost:3005
- IDM_PORT=3005
- IDM_HTTPS_ENABLED=false
- IDM_HTTPS_PORT=3443
- IDM_ADMIN_USER=admin
- IDM_ADMIN_EMAIL=admin#test.com
- IDM_ADMIN_PASS=test
secrets:
- my_secret_data
healthcheck:
test: curl --fail -s http://localhost:3005/version || exit 1
mysql-db:
restart: always
image: mysql:5.7
hostname: mysql-db
container_name: db-mysql
expose:
- "3306"
ports:
- "3306:3306"
networks:
default:
ipv4_address: 172.18.1.6
environment:
- "MYSQL_ROOT_PASSWORD_FILE=/run/secrets/my_secret_data"
- "MYSQL_ROOT_HOST=172.18.1.5"
volumes:
- mysql-db-sso:/var/lib/mysql
- ./mysql-data:/docker-entrypoint-initdb.d/:ro
secrets:
- my_secret_data
networks:
default:
ipam:
config:
- subnet: 172.18.1.0/24
volumes:
mysql-db-sso:
secrets:
my_secret_data:
file: ./secrets.txt
I have the Grafana application registered in Keyrock and has as callback http://172.18.1.4:3000/login. When I try to Sign-in in Grafana through Oauth it redirects me to the keyrock page to Sign-in, but when entering the credentials it returns me an invalid client_id, but it is the same one that returns Keyrock to me when obtaining the application information.
Is it possible that I lack something to configure or should it be done in another way?
Here is the working configuration for Keyrock 7.5.1 and Grafana 6.0.0
Grafana:
[auth.generic_oauth]
enabled = true
allow_sign_up = true
client_id = ${CLIENT_ID}
client_secret = ${CLIENT_SECRET}
scopes = permanent
auth_url = ${KEYROCK_URL}/oauth2/authorize
token_url = ${KEYROCK_URL}/oauth2/token
api_url = ${KEYROCK_URL}/user
App in Keyrock:
url - ${GRAFANA_ROOT_URL}
callback_url - ${GRAFANA_ROOT_URL}/login/generic_oauth
Token types - Permanent
So you need to fix env variable
GF_AUTH_GENERIC_OAUTH_API_URL
to
http://172.18.1.5:3005/user
and callback url
http://172.18.1.4:3000/login
to
http://172.18.1.4:3000/login/generic_oauth
and add oauth2 scopes
First of all I'm sorry if I'm not using the right terms to ask this question, but I'm not up to the terminology in place.
I have traefik running in a docker container and serving some services with the PathPrefix option, for instance, www.myserver.com/wordpress redirects to a docker container running wordpress.
But how do I get it to redirect to outside a docker container? Specifically, how do I get www.myserver.com to redirect to port 8080 in my machine to serve a service I have running there in the host OS (not in a docker container)?
This is my traefik.toml:
logLevel = "DEBUG"
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
compress = false
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[acme]
email = "mymail#mail.com"
storage = "acme.json"
entryPoint = "https"
onHostRule = true
#onDemand = true
[[acme.domains]]
main = "www.myserver.com"
[web]
address = ":8888"
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "www.myserver.com"
watch = true
exposedbydefault = false
And my docker-compose.yml for the traefik container:
version: "2"
services:
traefik:
image: traefik
network_mode: "host"
ports:
- "80:80"
- "443:443"
- "8888:8888"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${SERVER_DIR}/AppData/traefik:/etc/traefik/
- ${PWD}/acme.json:/acme.json
- ${PWD}/traefik.toml:/etc/traefik/traefik.toml
- ${PWD}/servers.toml:/etc/traefik/servers.toml
restart: never
With the new Traefik (v.2) you need to use a combination of labels and an external file, you can find below my working example.
In your docker compose you need to add the comands to define the external file and enable the provider
- "--providers.file=true"
- "--providers.file.filename=/etc/traefik/rules.toml"
Into your file (rules.toml) the routing to foward to your external service (be aware of the syntax, use the char to define the host ( ` ) )
example :
Docker-compose:
traefik:
image: "traefik:v2.0.0"
container_name: "traefik"
restart: always
command:
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myhttpchallenge.acme.httpchallenge=true"
- "--certificatesresolvers.myhttpchallenge.acme.httpchallenge.entrypoint=web"
- "--certificatesresolvers.myhttpchallenge.acme.email=xx#xx.com"
- "--providers.file=true"
- "--providers.file.filename=/etc/traefik/rules.toml"
- "--providers.docker=true"
- "--providers.file.watch=true"
ports:
- "80:80"
- "8080:8080"
- "443:443"
networks:
- proxy
environment:
- CF_API_EMAIL="xx"
- CF_API_KEY="xx"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./traefik/rules.toml:/etc/traefik/rules.toml"
Rules.toml
[http.routers]
# Define a connection between requests and services
[http.routers.nasweb]
rule = "Host(`nas.xxxx.com`)"
entrypoints = ["websecure"]
service = "nas"
[http.routers.nasweb.tls]
certResolver = "myhttpchallenge"
[http.services]
# Define how to reach an existing service on our infrastructure
[http.services.nas.loadBalancer]
[[http.services.nas.loadBalancer.servers]]
url = "http://192.168.0.165:80"
I've fiddled around and found the answer.
In traefik.toml add:
################################################################
# File configuration backend
################################################################
# Enable file configuration backend
# Optional
[file]
filename = "servers.toml"
# Enable watch file changes
watch = true
In docker-compose.yml change the volumes: to:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${SERVER_DIR}/AppData/traefik:/etc/traefik/
- ${PWD}/acme.json:/acme.json
- ${PWD}/traefik.toml:/etc/traefik/traefik.toml
- ${PWD}/servers.toml:/servers.toml
Add file servers.toml:
loglevel = "ERROR"
[backends]
[backends.nasweb]
[backends.nasweb.servers.nasweb]
url = "http://192.168.1.11:8080"
[frontends]
[frontends.domain]
backend = "nasweb"
[frontends.domain.routes.domain]
rule = "Host:www.myserver.com"