How dynamic map service name to ENV var - docker-compose

Example:
my-server:
image: my-server:latest
ports:
- 1234:1234
proxy:
image: lb:latest
environment:
- BACKEND=${VAR}??? # must be resolve as 'my-server'
The server name can be changed to any name, but the proxy has a entry-point script where the variable will be substituted in the BACKEND to config.

You can use a .env file to define your variable. This file will be placed in the same directory as your docker-compose.yml file.
When you run docker-compose, it will read this value and use it. Using your example, your .env file would look something like this:
VAR=my-server
and, the line:
- BACKEND=${VAR}??? # must be resolve as 'my-server'
would become just:
- BACKEND=${VAR}
or
BACKEND: ${VAR}

Related

How to break a long environment variable text into multiple lines in dotenv file (for docker-compose)

I have a regular docker-compose.yml
version: "3.9"
services:
test_service:
image: some_img:0.1.0
...
...
env_file:
- .env
...
...
here, I am loading environment variables using a .env file. In the file I have an environment variable with a long string value.
# .env
SPACE=test
SPECIFICS=packageA,packageB,packageC,packageD,...
I want SPECIFICS to parsed as above(SPECIFICS=packageA,packageB,packageC,packageD,... ) but would like to break it down into multiple lines for easy reading, may be like below (with or without indentation)
SPECIFICS=packageA,
packageB,
packageC,
packageD,
...
I do not want to use environment: section in docker-compose.yml because I am loading quite a few environment variables which will bloat the corresponding section.
Can anyone suggest a solution for this
CHeers,
DD.

how to set an environment variable when run - az container app compose create

It seems when my docker compose yml is run via "az containerapp compose create", environment variables are not picked up. Is there a way I can set an env variable so the command picks it up?
I'm seeing this error:
ERROR: The following field(s) are either invalid or missing. Invalid value: "${DOCKER_REGISTRY-}/sample-blazorapp": could not parse reference: ${DOCKER_REGISTRY-}/sample-blazorapp: template.containers.blazorapp.image.
I have set the variable with: export DOCKER_REGISTRY="myregistry"
And when I echo $DOCKER_REGISTRY, the value is returned. So in the bash session it is set (I tried powershell first, I thought that was the issue because $(envvar-) is bash syntax, however the error is the same.)
This is what I have in my compose file (alignment is correct in the file):
blazorapp:
container_name: "blazorapp"
image: ${DOCKER_REGISTRY-}sample-blazorapp
build:
context: .
dockerfile: BlazorApp/BlazorApp/Dockerfile
depends_on:
- redis
ports:
- "55000:443"
If I explicitly set the image name, i.e. not use an env var, then it works. i.e. this change to the image line works:
image: myregistry/sample-blazorapp
I also tried adding the forward slash, this makes no difference (as expected, it works fine without the slash when running docker compose up).
I can set it explicitly but that would be annoying. I feel like I'm missing something. Any help or guidance is greatly appreciated :)
If the image is defined like this into you docker compose file:
image: ${DOCKER_REGISTRY-}sample-blazorapp
then you must export using a slash at the end of the value:
export DOCKER_REGISTRY="myregistry/"
I discovered the issue, I was missing a colon.
Does not work (produces the error described in the question):
image: ${DOCKER_REGISTRY-}sample-blazorapp
Also does not work:
image: ${DOCKER_REGISTRY-mydefault}sample-blazorapp
Add the magic : in and it works:
image: ${DOCKER_REGISTRY:-}sample-blazorapp
Also works:
image: ${DOCKER_REGISTRY:-mydefault}sample-blazorapp

Unable to set file secrets passwords on docker-compose

I want to access a secret file with docker-compose. I follow many tutorials and reproduce the structure. The container is launched but when I need to log in, the right string to enter is "/run/secrets/sec_pgadmin_default_email". I want the login to be the one inside my secret file: admin. In this secret file pgadmin_default_email.txt, there is only the word admin. I don't know if the structure of this file is good. Should it be a dictionary or something else ?
version: '3.8' services: pgadmin4:
build:
context: ./dockerfiles/
dockerfile: dockerfile_pgadmin4
image: img_pgadmin
container_name: cont_pgadmin4
ports:
- "80:80"
secrets:
- sec_pgadmin_default_email
- sec_pgadmin_default_password
environment:
- PGADMIN_LISTEN_PORT=80
- PGADMIN_DEFAULT_EMAIL=/run/secrets/sec_pgadmin_default_email
- PGADMIN_DEFAULT_PASSWORD=/run/secrets/sec_pgadmin_default_password
secrets:
sec_pgadmin_default_email:
file: ./.secrets/pgadmin_default_email.txt
sec_pgadmin_default_password:
file: ./.secrets/pgadmin_default_password.txt
PS I just use the command docker-compose up -d, without swarn things or else. I don't define external secrets and I prefer not. Is it possible ?
the environment variable will be set to the path of the file containing the actual value. Not the actual value itself.
For example, in Go you could do something like:
var pgAdminDefaultEmailFile = os.Getenv("PGADMIN_DEFAULT_EMAIL") // get the file
pgAdminDefaultEmailValue, err := ioutil.ReadFile(pgAdminDefaultEmailFile) // get the value
if err != nil {
log.Fatal(err)
}
pgAdminDefaultEmail = string(pgAdminDefaultEmailValue) // parse out the value

Environment precedence and ports

I have the following in my Dockerfile:
. . .
ENV SSL_PORT=443
. . .
EXPOSE ${SSL_PORT}
. . .
And the following in a docker-compose.override.yml file calling that image:
environment:
SSL_PORT: $SSL_PORT
ports:
- "${SSL_PORT}:${SSL_PORT}"
If I do that
WARNING: The SSL_PORT variable is not set. Defaulting to a blank string.
ERROR: The Compose file '.\docker-compose.override.yml' is invalid because:
services.ssl.ports contains an invalid type, it should be a number, or an object
If I set it in the .env file, the container is built.
Is there any way I can set the value of SSL_PORT in docker-compose.override.yml AND use that same value?
There are only two ways to set value of SSL_PORT variable into docker-compose.override.yml:
1. Declaring default environment variables in an environment file named .env placed in the folder where the docker-compose command is executed.
2. Add SSL_PORT variable into environment variables when you execute docker-compose command. It would be either:
SSL_PORT=443 docker-compose -f docker-compose.override.yml up
or
export SSL_PORT=443
docker-compose -f docker-compose.override.yml up

Copy file to ansible host with custom variables substituted

I'm working on an ansible-playbook which should help to generate build agents for a continuous delivery pipeline. Among other issues, I'll need to install an oracle client on such an agent. I want to do something like
- name: "Provide response file"
copy: src=/custom.rsp dest=/opt/oracle
Within the custom.rsp file I've got some variables to be substituted. Normally, one could do it with a separate shell command like this:
- name: "Substitute Vars"
shell: "sed 's|<PARAMETER>|<VALUE>|g' -i /opt/oracle/custom.rsp"
I don't like it, though. There should be a more convinient way to do this. Anybody giving me a hint?
You want to be using a template rather than copying a static file.
Also, when using the copy or template modules, the dest parameter is a full path AND filename, not just a path. So if you want to end up with a copy of custom.rsp in the directory /opt/oracle then you need to do this:
- name: "Provide response file"
template: src=/custom.rsp dest=/opt/oracle/custom.rsp
I'm going to extend Bruce's answer with an example:
This is part of my inventory.yaml:
kafka_stage:
children:
kafka_with_zookeeper_stage:
kafka_only_stage:
vars:
zookeeper_hosts: "kafka-stage01:2181,kafka-stage02:2181,kafka-stage03:2181"
kafka_with_zookeeper_stage:
hosts:
kafka-stage01:
broker_id: 0
kafka-stage02:
broker_id: 1
vars:
services:
kafka:
zookeeper:
This is part of a configuration file:
# The id of the broker. This must be set to a unique integer for each broker.
broker.id={{ broker_id }}
# {{ zookeeper_hosts }}
advertised.listeners=PLAINTEXT://{{ ansible_host }}:9092
# {{ services }}
This command in a playbook:
- name: Copy to Host
ansible.builtin.template:
src: my_configfile.properties
dest: /tmp/hejsan.properties
Gave me this on the remote host kafka-stage02:
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=1
# kafka-stage01:2181,kafka-stage02:2181,kafka-stage03:2181
advertised.listeners=PLAINTEXT://kafka-stage02:9092
# {'kafka': None, 'zookeeper': None}