I am trying to perform variable substitution inside of a .env file but have not had any luck so far.
I've been looking though the docker-compose documentation and have not found anything mentioning this (or any examples online) but it seems like something that would be surprising if not possible.
What I am talking about is doing something like this in my .env file:
// .env
SOME_LOCATION=/path/to/some/location
CONFIG_FILE=${SOME_LOCATION}/config
CONSTANT_FILE=${SOME_LOCATION}/constants
(This example makes CONFIG_FILE equal to the string $${SOME_LOCATION}/config and same thing happens with CONSTANT_FILE)
I realize that this is possible inside of the compose.yml file with syntax like this but can it be done just inside the .env file?
I'm using docker-compose version 1.24.1 if it's not possible then I will just copy past these kinds of things but it always feels dirty copying the same values through your configuration.
You can change or add .env variable with export command in shell scripting.
File path like this;
-project
-> docker-compose.yml
-> .env
-> start.sh
docker-compose.yml
services:
jira:
image: atlassian/jira-software
volumes:
- ${CONFIG_FILE}:/var/atlassian/application-data/jira
ports:
- 8080:8080
volumes:
jira_vol:
external: false
start.sh
export "CONFIG_FILE=${SOME_LOCATION}/jira_vol"
docker-compose up -d
.env
SOME_LOCATION=./volumes
Finally run this command;
sh start.sh
start.sh file will add CONFIG_FILE variable and run docker compose. And you can use CONFIG_FILE and SOME_LOCATION in docker-compose.yml.
Related
I just want to rename a few files, without overriding the commands inside the wordpress image that the docker is pulling in.
Inside the docker-compose.yml I tried using 'command' and 'entrypoint' to run bash commands, both basically interrupt what's happening inside the image and it all fails.
you have three main ways to run a command after the container starts:
with docker exec -d someContainer some command from the command line,
with CMD ["some", "command"] from your Dockerfile
with command: some command from a docker-compose file
if none of these is working for you, probably, you are doing something wrong. A common mistake is using multiple command in your docker-compose file, like so:
version: '3.8'
services:
someService:
command: a command
command: another command
this doesn't work, because the last command overrides the commands above, what you should do is concatenate the commands:
version: '3.8'
services:
someService:
command: a command && another command
take a look at this question.
edit: one thing i forgot to include is that the same behavior above is true to CMD in your Dockerfile, you can't do this:
CMD ["some", "command"]
CMD ["another", "command"]
instead, you should concatenate the commands, just like the docker-compose:
CMD ["some", "command", "&&", "another", "command"]
but this is very boring if you have a lot of commands, so an alternative is to use a shell script with all the commands you need and execute it in your Dockerfile:
#!/bin/sh
# bash file with your commands
run wordpress && rename files && do something else
# later in your Dockerfile
CMD ["sh", "/path/to/file.sh"]
see this question
As you haven't provided any code it's hard to say, but also, maybe you can use RUN command to rename as the last command(just before the CMD if you are using it) in your Dockerfile to rename these files at build time(what IMHO makes more sense because this is kind of thing you should do when you are building your images). So if you want more help, please, include your code too.
I have a few Dockerfiles that are dependant on a "pat" (Personal Access Token) file to be able to access a private nuget feed. I have taken some inspiration from somakdas to get this working.
To run my single Dockerfile I first create a "pat" file containing my token and build with docker build -f Services/User.API/Dockerfile -t userapi:dev --secret id=pat,src=pat .
This works as intended, but my issue is getting this to work using a docker-compose.yml file.
First I took a look at using docker-compose secrets, but it came to my attention that docker-compose secrets are access at runtime, not build-time. https://github.com/docker/compose/issues/6358
So now I'm trying to create a volume containing my pat file but I get cat: /pat: No such file or directory when the command RUN --mount=type=secret... is running. This may not be secure but it will only be running locally.
My Dockerfile
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | bash
WORKDIR /src
COPY ["User.API.csproj", "/Services/User.API"]
RUN --mount=type=secret,id=pat,dst=/pat export ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{\"endpointCredentials\": [{\"endpoint\":\"<feed>\", \"username\":\"<user>\", \"password\":\"`cat /pat`\"}]}" \
&& dotnet restore "User.API.csproj" \
&& unset VSS_NUGET_EXTERNAL_FEED_ENDPOINTS
...
My docker-compose.yml
services:
user.api:
container_name: User.API
image: ${DOCKER_REGISTRY-}userapi
build:
context: .
dockerfile: Services/User.API/Dockerfile
networks:
- app_network
volumes:
- ./pat:/app/src/pat
Am I only able to access docker-compose volumes after the Dockerfile is built?
I solved this by attacking the problem in a different way. As the main goal was to get this working locally I created Dockerfile.Local and docker-compose.local.yml. Together with this I created an .env file containing the "pat".
The docker-compose.local.yml passes the "pat" as an argument to the Dockerfile.Local where it's used. I also discarded --mount=type=secret and set the value to VSS_NUGET_EXTERNAL_FEED_ENDPOINTS directly.
.env file:
PAT_TOKEN=<personal access token>
docker-compose.local.yml:
services:
user.api:
container_name: User.API
image: ${DOCKER_REGISTRY-}userapi
build:
context: .
dockerfile: Services/User.API/Dockerfile
args:
- PAT=${PAT_TOKEN}
networks:
- app_network
volumes:
- ./pat:/app/src/pat
Dockerfile.Local:
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | bash
WORKDIR /src
COPY ["User.API.csproj", "/Services/User.API"]
ARG PAT
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{\"endpointCredentials\": [{\"endpoint\":\"<feed>\", \"username\":\"<user>\", \"password\":\"${PAT}\"}]}" \
&& dotnet restore "User.API.csproj" \
&& unset VSS_NUGET_EXTERNAL_FEED_ENDPOINTS
...
Note: The .env file was added to .gitignore because it is containing sensitive information. We don't want that in our repository.
I have added a few variables in a .env file.
e.g.
POSTGRES_USER=user
POSTGRES_PASSWORD=pass
Postgres does not create a user with those credentials.
I can not see them when I run docker exec -it $(docker ps | ack postgre | cut -f1 -d\ ) env either.
Why isn't the environnment variables I set in .env honored when I start docker-compose?
Turns out that one has to tell docker-compose that an .env file exists:
e.g.
web:
env_file:
- web-variables.env
https://docs.docker.com/compose/environment-variables/#the-envfile-configuration-option
I tried to use a .env file to make environment variable, but it doesn't work.
These are my steps:
version "3"
services:
web:
image: php-fpm:5.6.30
env_file:
- .env
This is .env file
TEST_ENV="HELLO WORLD"
It doesn't work when I start the container:
var_dump(getenv("TEST_ENV")); // output NULL
For me it seems to work. Maybe this can help you:
├── docker-compose.yaml
├── .env
└── myphp
├── Dockerfile
└── script.php
My .env file
TEST_ENV="HELLO WORLD"
My docker-compose.yaml:
version: "3"
services:
web:
build: ./myphp
env_file: .env
So my docker-compose.yaml will build the image myphp. The dockerfile looks like this:
FROM php:5.6.30-fpm
COPY script.php /var/script.php
My script.php
<?php
var_dump(getenv('TEST_ENV'));
exit;
Than I perform docker-compose up -d --build. This will build my image and add the php script in it and it will run a container instance of that image.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
15f0289bfbe8 test_web "docker-php-entryp..." 3 seconds ago Up 1 second 9000/tcp test_web_1
I'm accessing my container
$ docker exec -it 15f0289bfbe8 bash
And I'm going the the /var folder where I've put my script (check dockerfile) and I'm exexcuting it + also just printing env var:
root#15f0289bfbe8:/var/www/html# cd /var/
root#15f0289bfbe8:/var# ls
backups cache lib local lock log mail opt run script.php spool tmp www
root#15f0289bfbe8:/var# php -f script.php
string(13) ""HELLO WORLD""
root#15f0289bfbe8:/var# echo $TEST_ENV
"HELLO WORLD"
root#15f0289bfbe8:/var#
I'm trying to pass a host variable to a Dockerfile when running docker-compose build
I would like to run
RUN usermod -u $USERID www-data
in an apache-php7 Dockerfile. $USERID being the ID of the current host user.
I would have thought that the following might work:
commandline
EXPORT USERID=$(id -u); docker-compose build
docker-compose.yml
...
environment:
- USERID=$USERID
Dockerfile
ENV USERID
RUN usermod -u $USERID www-data
But no luck yet.
For Docker in general it is generally not possible to use host environment variables during the build phase; this is because it is desirable that if you run docker build and I run docker build using the same Dockerfile (or Docker Hub runs `docker build with the same Dockerfile), we end up with the same image, regardless of our local environment.
While passing in variables at runtime is easy with the docker command line (using -e <var>=<value>), it's a little trickier with docker-compose, because that tool is designed to create self-contained environments.
A simple solution would be to drop the host uid into an environment file before starting the container. That is, assuming you have:
version: "2"
services:
shell:
image: alpine
env_file: docker-compose.env
command: >
env
You can then:
echo HOST_UID=$UID > docker-compose.env; docker-compose up
And the HOST_UID environment variable will be available to your
container:
Recreating vartest_shell_1
Attaching to vartest_shell_1
shell_1 | HOSTNAME=17423d169a25
shell_1 | HOST_UID=1000
shell_1 | HOME=/root
vartest_shell_1 exited with code 0
You would then to have something like an ENTRYPOINT script that
would set up the container environment (creating users, modifying file
ownership) to operate correctly with the given UID.