Docker Toolbox Windows - Invalid volume specification - docker-compose

EDIT: The problem is the COMPOSE_CONVERT_WINDOWS_PATHS environment variable isn't getting set. I tried adding it to the Dockerfile and to a .env file but its not being set. I can't set any new environment variables. Its like its building it from a cached Dockerfile.
I was running docker-composer up after deleting the container, but this didn't recreate a new container, even when adding --force-recreate, it created the container from the old Dockerfile. So I ran docker build ./ instead, and it built a new container, it reinstalled all the packages, and it said that it was setting the new environment variables, but in the finished container, when I check kinematic, I see the new environment variables aren't set. But all of the old environment variables that were set when I built the container for the first time, they're all set. I don't know why this is happening, why I can't create a new container from the updated Dockerfile.
I've found a number of threads about this error, it happens when trying to mount a volume in Windows docker-tools. When I run docker-compose up, I get this error:
ERROR: for web Cannot create container for service web: Invalid bind mount spec "C:\\path\\to\\project:/app:rw": Invalid volume specification: 'C:\path\to\project:/app:rw'
[31mERROR[0m: Encountered errors while bringing up the project.
In the docker-compose file I have this:
volumes:
- ./:/app
I tried setting this environment variable:
COMPOSE_CONVERT_WINDOWS_PATHS=1
to 1, as that worked for some people in another thread but its not working in this case. I tried adding an absolute path in linux like format:
volumes:
- /c/path/to/project:/app
and I get this error:
ERROR: for web Cannot create container for service web: create \c\path\to\project: "\\c\\path\\to\\project" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
Notice it reversed the forward slashes. Changing:
ENV COMPOSE_CONVERT_WINDOWS_PATHS=0
to 0 like that didn't change anything.
If I give a Windows format absolute directory, I get the first error again. Both of those errors, I get them if I point to a directory that doesn't exist.
Has anyone here figured out a way to resolve this?

So, if this problem exists, you should create new Windows environment variable called COMPOSE_CONVERT_WINDOWS_PATHS and set it to 1. Or you can create .env file in the path docker-compose.yml is placed with following content:
COMPOSE_CONVERT_WINDOWS_PATHS=1
..
cat >.env
COMPOSE_CONVERT_WINDOWS_PATHS=1

It looks like your COMPOSE_CONVERT_WINDOWS_PATHS=1 environment variable is not read by docker cli (powershell). you can try to run Get-childItem Env: command from your powershell console to get the list of environment variable which has been set in your powershell console. the output is like below:
Name Value
---- -----
COMPOSE_CONVERT_WINDOWS_PATHS 1
COMSPEC C:\WINDOWS\system32\cmd.exe
DOCKER_CERT_PATH C:\Users\devsa\.docker\machine\machines\default
DOCKER_HOST tcp://192.168.99.100:2376
DOCKER_TLS_VERIFY 1
...
If you don't see the COMPOSE_CONVERT_WINDOWS_PATHS=1 variable in your powershell console, it means the variable has not been set yet. To set the environment variable via powershell console, try this command:
$env:COMPOSE_CONVERT_WINDOWS_PATHS=1
it will set the environment variable in your power shell console check this powershell documentation for full documentation. Try to run the Get-childItem Env: command again to check whether the variable has been already set or not. if the variable has been set, you can try to run docker-compose up -d again.

After setting the environment variable to COMPOSE_CONVERT_WINDOWS_PATHS=1 try below setting in docker-compose
- ./app-demo:/dockermount under volumes
It worked for me

Related

How to use environment variable in Dockerfile RUN

I've read several tutorials about this and it seems like this code should work. I want to use RUN in my Dockerfile to write environment variables to a config file. The Dockerfile is simply:
FROM php:7.4-apache
RUN echo "I am ${USERNAME}" > /home/dockerfile.log
I am starting it with this docker-compose.yml
version: "3"
services:
web:
build:
context: .
environment:
- USERNAME=larry
After starting with docker-compose up -d the log file just says I Am so the environment variable is empty. What's wrong here? If I run docker-compose config I can see the variable shown correctly there.
The Run command is executed at build time (I.e. when building the image ) while the environment variable in your docker-compose is added to the container at run time. So when the Run command is executed the environment variable has not been populated yet. If you want to pass a parameter at build time, I would suggest using a build argument instead. In your compose file use arg instead of environment

current directory variable substitution in docker-compose for windows 10

I am trying to create absolute paths in my docker-compose using variable substitution for the path up to the current directory. I pass these paths as environment variables in my container.
More precisely I try to pass it via the cd command and not via the PWD env var. It is supposed to be possible declaring the env var in the .env (https://github.com/docker/compose/issues/4081).
Unfortunately I cannot get the substitution to work.
I tried the different approaches:
with the declaration of the var as a environment variable in a .env OR directly in the environment list
Running with powershell and command prompt
using %cd% and $pwd with both shells
escape $ and %, doubling them down with extra $ or %
Relevant piece of the docker-compose:
version: "3.2"
services:
jupyter:
environment:
- WORKING_DIR=%cd% # OR $(pwd) OR $VAR
and when I use the .env:
VAR=%cd% or ${pwd}
When I 'set' my environment variables in my container, it just shows the command I used (%cd%, etc) as if they were strings.
I do not quite understand where is the docker-compose formated/executed: in the shell I use to call docker-compose, in the linux image where my docker daemon runs, etc ?
EDIT:
I tried on a MacOS version and I still have the problem ! (using adequate bash variable substitution)
Not a complete solution, but perhaps something you can script:
If you specify DOCUMENT_ROOT as a variable in the docker compose file
environment:
- WORKING_DIR=${DOCUMENT_ROOT}
And have an .env file in your working directory that contains
DOCUMENT_ROOT=./
, then you can use the local .env file by running
docker-compose -f /path/to/docker-compose.yml up
https://docs.docker.com/compose/compose-file/#env_file

Setting up Clusters and work nodes - How do I set KUBECONFIG variable in Windows 10?

I using following link to create :
https://console.bluemix.net/docs/containers/cs_tutorials.html#cs_cluster_tutorial
In Lesson3, Step 3 "Verify that the KUBECONFIG environment variable is set properly.", I am unable to set the variable KUBECONFIG.
I am able to execute the following command:
ibmcloud ks cluster-config clusternameabc
I am getting following message
The configuration for clusternameabc was downloaded successfully. Export environment variables to start using Kubernetes, with the following message:
SET KUBECONFIG=C:\Users\AAA.bluemix\plugins\container-service\clusters\customernameabc\kube-config-hou02-clusternameabc.yml
When I run the above SET command in powershell, I just get the prompt back with environment variable not set.
When I do
$Env:KUBECONFIG=C:\Users\AAA.bluemix\plugins\container-service\clusters\customernameabc\kube-config-hou02-clusternameabc.yml
Notepad gets opened and this file is shown. I want to just set the environment variable and unable to. Can someone please tell me what I am doing wrong? I tried searching and could not find answers and IBM documentation has only Mac examples.
In Powershell
1) $env:KUBECONFIG = "C:\Users\AAA.bluemix\plugins\container-service\clusters\customernameabc\kube-config-hou02-clusternameabc.yml"
2) ls env:KUBECONFIG
Slightly easier to remember.
Based on my tests, you have to wrap the value of the environment environment in double quotes, like so:
$Env:KUBECONFIG="C:\Users\AAA.bluemix\plugins\container-service\clusters\customernameabc\kube-config-hou02-clusternameabc.yml"
Then, you can check the environment variables in PowerShell via this command:
Get-ChildItem Env:
Try this:
$Env:KUBECONFIG = ibmcloud cs cluster-config --export clusternameabc | Select -First 1 | % {$_.replace("SET KUBECONFIG=","")}

Invalid mount config for type "bind": bind source path does not exist when trying to run container on Docker for Windows

I'm trying to run the Prometheus sample at https://docs.docker.com/engine/admin/prometheus/#use-prometheus using Docker on Windows. I'm executing the following command:
docker service create --replicas 1 --name my-prometheus `
--mount type=bind,source="C:/temp/prometheus.yml",destination=/etc/prometheus/prometheus.yml `
--publish published=9090,target=9090,protocol=tcp `
prom/prometheus
The error is: invalid mount config for type "bind": bind source path does not exist
I'm pretty certain the file exists (Test-Path with PowerShell returns true).
Any idea what I'm doing wrong?
Update
Running the same command with mount type = volume gives me the following error:
C:/temp/prometheus.yml" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path"
I am passing an host directory and "C:/temp/prometheus.yml" is an absolute path.
You may run into this on Windows after you change your Windows account password.
Open Docker for Windows Settings.
On Shared Drives tab click Reset credentials...
Share your drive once again.
i got same error happened to me. as below picture, uncheck "Use Docker Compose V2" and it works for me. I wanted to share in case this might be the solution for somebody

docker compose environment variable for command

I am having troubles in passing environment variables to my custom image via the compose command option:
My compose file:
---
version: '2'
services:
myservice:
image: mycustomimage_lms
environment:
CONF_HOME: /opt/apps-java/
APP_ENV: dev
UUID: me1
command: -Dconfig.home=${CONF_HOME} -Dcomponent.name=LMS -Denv=${APP_ENV} -Duser.dir=/tmp/ -DLMS_UUID=${UUID} -jar /opt/apps-java/my.jar
ports:
- "9060"
volumes:
- ./:/opt/apps-java/
- ./:/var/logs/apps-logs/
- ./:/tmp/data
My image is just a custom jre image which has an entrypoint set to a shell script that accepts jvm arguments. My run.sh that is called from enrtypoint
#!/bin/sh
export JAVA_HOME="/usr/java/latest/"
exec $JAVA_HOME/bin/java $#
I need to pass values to command at runtime since I can then use my image for a lot of other jars and just changing parameters to my image.
This is what i get:
$> docker-compose up
WARNING: The CONF_HOME variable is not set. Defaulting to a blank string.
WARNING: The APP_ENV variable is not set. Defaulting to a blank string.
WARNING: The UUID variable is not set. Defaulting to a blank string.
I have also
gone through couple of answers such as :
Docker Compose - Command using Container Environment Variable
and
Docker-compose environment variables
but could not get it working. Any directions please?
The variables are being read by Compose when the file is parsed. But setting environment only provides values to the container, not to the file parsing.
If you're trying to pass those variables into the container, you need to escape them in the command using an extra $
-Dconfig.home=$${CONF_HOME} -Dcomponent.name=LMS -Denv=$${APP_ENV} -Duser.dir=/tmp/ -DLMS_UUID=$${UUID
If you're just trying to use variables in the Compose file, you need to put those variables into an .env file.
See https://docs.docker.com/compose/compose-file/compose-file-v3/#variable-substitution for the full documentation
Escaping the $ so the variable is not substituted immediately but later in the container environment is the trick, as the accepted answer does in the the docker-compose.yml.
I just wanted to add that in case you pass a command to the docker-compose call, you need to escape the character in the shell, i.e. with \
docker-compose run --rm myservice "-Dconfig.home=\${CONF_HOME} -Dcomponent.name=LMS -Denv=\${APP_ENV} -Duser.dir=/tmp/ -DLMS_UUID=\${UUID} -jar /opt/apps-java/my.jar"