MIx environment variables from kubernetes deployment and docker image - kubernetes

I am trying to pass an environment variable in my deployment that should define a prefix based on a version number:
env:
- name: INDEX_PREFIX
value: myapp-$(VERSION)
$(VERSION) is not defined in my deployment but is set in the docker image used by the pod.
I tried to use both $() and ${} but VERSION is not interpolated in the environment of my pod. In my pod shell doing export TEST=myapp-${VERSION} does work though.
Is there any way to achieve what I am looking for? ie setting an environment variable in my deployment that reference an environment variable set in the docker image?

VERSION is an environment variable of the docker image. So you can assign it a value either inside the container or by passing
env:
- name : VERSION
value : YOUR-VALUE
In your case, VERSION is either set by a script inside the docker container or in the Dockerfile.
You can do :
In the Dockerfile, adding ENV INDEX_PREFIX myapp-${VERSION}
Adding a script to your entrypoint as
export INDEX_PREFIX=myapp-${VERSION}
In case you can't modify Dockerfile, you can try to :
Get the image entrypoint file from the docker image (ie: /IMAGE-entrypoint.sh) and the image args(ie: IMAGE-ARGS). you can use docker inspect IMAGE.
Override the container command and args in the pod spec using a script.
command:
- '/bin/sh'
args:
- '-c'
- |
set -e
set -x
export INDEX_PREFIX=myapp-${VERSION}
IMAGE-entrypoint.sh IMAGE-ARGS
k8s documentation : https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
Hope it could help you.

Related

how to set an environment variable when run - az container app compose create

It seems when my docker compose yml is run via "az containerapp compose create", environment variables are not picked up. Is there a way I can set an env variable so the command picks it up?
I'm seeing this error:
ERROR: The following field(s) are either invalid or missing. Invalid value: "${DOCKER_REGISTRY-}/sample-blazorapp": could not parse reference: ${DOCKER_REGISTRY-}/sample-blazorapp: template.containers.blazorapp.image.
I have set the variable with: export DOCKER_REGISTRY="myregistry"
And when I echo $DOCKER_REGISTRY, the value is returned. So in the bash session it is set (I tried powershell first, I thought that was the issue because $(envvar-) is bash syntax, however the error is the same.)
This is what I have in my compose file (alignment is correct in the file):
blazorapp:
container_name: "blazorapp"
image: ${DOCKER_REGISTRY-}sample-blazorapp
build:
context: .
dockerfile: BlazorApp/BlazorApp/Dockerfile
depends_on:
- redis
ports:
- "55000:443"
If I explicitly set the image name, i.e. not use an env var, then it works. i.e. this change to the image line works:
image: myregistry/sample-blazorapp
I also tried adding the forward slash, this makes no difference (as expected, it works fine without the slash when running docker compose up).
I can set it explicitly but that would be annoying. I feel like I'm missing something. Any help or guidance is greatly appreciated :)
If the image is defined like this into you docker compose file:
image: ${DOCKER_REGISTRY-}sample-blazorapp
then you must export using a slash at the end of the value:
export DOCKER_REGISTRY="myregistry/"
I discovered the issue, I was missing a colon.
Does not work (produces the error described in the question):
image: ${DOCKER_REGISTRY-}sample-blazorapp
Also does not work:
image: ${DOCKER_REGISTRY-mydefault}sample-blazorapp
Add the magic : in and it works:
image: ${DOCKER_REGISTRY:-}sample-blazorapp
Also works:
image: ${DOCKER_REGISTRY:-mydefault}sample-blazorapp

Set Variables for Airflow during Helm install

I'm installing Airflow on kind with the following command:
export RELEASE_NAME=first-release
export NAMESPACE=airflow
helm install $RELEASE_NAME apache-airflow/airflow --namespace $NAMESPACE \
--set images.airflow.repository=my-dags \
--set images.airflow.tag=0.0.1 \
--values env.yaml
Andthe file env.yaml looks like the below:
env:
- name: "AIRFLOW_VAR_KEY"
value: "value_1"
But from the Web UI (when I go to Admins --> Variables), these credentials don't appear there.
How do I pass these credentials during helm install? Thanks!
UPDATE: It turns out that the environment variable was set successfully. However it doesn't show up on the Web UI
i am not sure how your full env.yaml file is
but to set the environment variables in Airflow
## environment variables for the web/scheduler/worker Pods (for airflow configs)
##
## WARNING:
## - don't include sensitive variables in here, instead make use of `airflow.extraEnv` with Secrets
## - don't specify `AIRFLOW__CORE__SQL_ALCHEMY_CONN`, `AIRFLOW__CELERY__RESULT_BACKEND`,
## or `AIRFLOW__CELERY__BROKER_URL`, they are dynamically created from chart values
##
## NOTE:
## - airflow allows environment configs to be set as environment variables
## - they take the form: AIRFLOW__<section>__<key>
## - see the Airflow documentation: https://airflow.apache.org/docs/stable/howto/set-config.html
##
## EXAMPLE:
## config:
## ## Security
## AIRFLOW__CORE__SECURE_MODE: "True"
## AIRFLOW__API__AUTH_BACKEND: "airflow.api.auth.backend.deny_all"
Reference file
and after that you have to run your command and further your DAG will be able to access the variables.
Helm documentation : https://github.com/helm/charts/tree/master/stable/airflow#docs-airflow---configs
Make sure you are configuring section : airflow.config
Okay, so I've figured this one out: The environment variables are set just nicely on the pods. However, this will not appear on the Web UI.
Workaround: to make it appear in the web UI, I will have to go into the Scheduler pod and import the variables. It can be done with a bash script.
# Get the name of scheduler pod
export SCHEDULER_POD_NAME="$(kubectl get pods --no-headers -o custom-columns=":metadata.name" -n airflow | grep scheduler)"
# Copy variables to the scheduler pod
kubectl cp ./variables.json airflow/$SCHEDULER_POD_NAME:./
# Import variables to scheduler with airflow command
kubectl -n $NAMESPACE exec $SCHEDULER_POD_NAME -- airflow variables import variables.json

Kubernetes command arguments being ignored

when running specific command from linux terminal command is the following:
/opt/front/arena/sbin/ads_start ads -db_server vmazfassql01 -db_name Test1
In regular docker compose yaml file we define it like this:
ENTRYPOINT ["/opt/front/arena/sbin/ads_start", "ads" ]
command: ["-db_server vwmazfassql01","-db_name Test1"]
Then I tried to convert it to Kubernetes
command: ["/opt/front/arena/sbin/ads_start","ads"]
args: ["-db_server vwmazfassql01","-db_name Test1"]
or without quotes for args
command: ["/opt/front/arena/sbin/ads_start","ads"]
args: [-db_server vwmazfassql01,-db_name Test1]
but I got errors for both cases:
Unknown parameter value '-db_server vwmazfassql01'
Unknown parameter value '-db_name Test1'
I then tried to remove dashes from args but then it seems those values are being ignored and not set up. During the Initialization values process, during the container start, those properties seem to have they default values e.g. db_name: "ads". At least that is how it is printed out in the log during the Initialization.
I tried few more possibilities:
To define all of them in command:
command:
- /opt/front/arena/sbin/ads_start
- ads
- db_server vwmazfassql01
- db_name Test1
To define them in little bit different way:
command: ["/opt/front/arena/sbin/ads_start","ads"]
args:
- db_server vwmazfassql01
- db_name Test1
command: ["/opt/front/arena/sbin/ads_start","ads"]
args: [db_server vwmazfassql01,db_name Test1]
again they are being ignored, and not being set up.
Am I doing something wrong? How I can make some workaround for this? Thanks
I would try separating the args, following the documentation example (https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#run-a-command-in-a-shell)
Something like:
command: ["/opt/front/arena/sbin/ads_start", "ads"]
args: ["-db_server", "vwmazfassql01", "-db_name", "Test1"]
Or maybe, it would work even like this and it looks more clean:
command: ["/opt/front/arena/sbin/ads_start"]
args: ["ads", "-db_server", "vwmazfassql01", "-db_name", "Test1"]
This follows the general approach of running an external command from code (a random example is python subprocess module) where you specify each single piece of the command that means something on its own.

How to run a python function or script somescript.py on the KubernetesPodOperator in airflow?

I am running a Celery Executor and I'm trying to run some python script in the KubernetesPodOperator. Below are examples of what I have tried that didn't work. What am I doing wrong?
Running sctipt
org_node = KubernetesPodOperator(
namespace='default',
image="python",
cmds=["python", "somescript.py" "-c"],
arguments=["print('HELLO')"],
labels={"foo": "bar"},
image_pull_policy="Always",
name=task,
task_id=task,
is_delete_operator_pod=False,
get_logs=True,
dag=dag
)
Running function load_users_into_table()
def load_users_into_table(postgres_hook, schema, path):
gdf = read_csv(path)
gdf.to_sql('users', con=postgres_hook.get_sqlalchemy_engine(), schema=schema)
org_node = KubernetesPodOperator(
namespace='default',
image="python",
cmds=["python", "somescript.py" "-c"],
arguments=[load_users_into_table],
labels={"foo": "bar"},
image_pull_policy="Always",
name=task,
task_id=task,
is_delete_operator_pod=False,
get_logs=True,
dag=dag
)
The script somescript.py must be in Docker image.
Step-1: let's create a image https://docs.docker.com/develop/develop-images/dockerfile_best-practices/.
FROM python:3.8
# copy requirement.txt from local to container
COPY requirements.txt requirements.txt
# install dependencies into container (geopandas, sqlalchemy)
RUN pip install -r requirements.txt
# copy the python script from local to container
COPY somescript.py somescript.py
ENTRYPOINT [ "python", "somescript.py"]
Step-2: Build and push the image into public Docker repository https://hub.docker.com.
NB: kubernetes_pod_operator looks for image from public docker repo
# build image
docker build -t my-python-img:latest .
# test if your image works perfectly
docker run my-python-img:latest
# push image.
docker tag my-python-img username/my-python-img
docker push username/my-python-img
docker pull username/my-python-img
step-3: Lest's create k8s task.
p = KubernetesPodOperator(
namespace='default',
image='username/my-python-img:latest',
labels={'dag-id': dag.dag_id},
name='airflow-my-image-pod',
task_id='load-users',
in_cluster=False, #False: local, True: cluster
cluster_context='microk8s',
config_file='/usr/local/airflow/include/.kube/config',
is_delete_operator_pod=True,
get_logs=True,
dag=dag
)
If you don't understand where configuration file comes from, look here: https://www.astronomer.io/docs/cloud/stable/develop/kubepodoperator-local.
Finally: I want to mention something important when working with databases (credentials). Kubernetes offers the use secret to secure sensitive information. https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html
KubernetesPodOperator launches a Kubernetes pod that runs a container as specified in the operator's arguments.
First Example
In the first example, the following happens:
KubernetesPodOperator instructs K8s to lunch a pod and prepare to run a container in it using the python image (the image parameter) from hub.docker.com (the default image registry)
ENTRYPOINT of the python image is replaced by ["python", "somescript.py" "-c"] (the cmd parameter)
CMD of the python image is replaced by ["print('HELLO')"] (the arguments parameter)
...
The container is run
So, the complete command that is run in the container is
python somescript.py -c print('HELLO')
Obviously, the official Python image from Docker Hub does not have somescript.py in its working directory. Even if did, it probably would have been not the one that you wrote. That is why the command fails with something like:
python: can't open file 'somescrit.py': [Errno 2] No such file or directory
Second Example
In the second example, pretty much the same happens as in the first example, but the command that is run in the container (again based on the cmd and arguments parameters) is
python somescript.py -c None
(None is the string representation of the load_users_into_table()'s return value)
This command fails, because of the same reasons as in the first example.
How It Could be Done (a Sketch)
You could build a Docker image with somescript.py and all its dependencies. Push the image to an image registry. Specify the image, ENTRYPOINT, and CMD in the corresponding parameters of KubernetesPodOperator.

Environment precedence and ports

I have the following in my Dockerfile:
. . .
ENV SSL_PORT=443
. . .
EXPOSE ${SSL_PORT}
. . .
And the following in a docker-compose.override.yml file calling that image:
environment:
SSL_PORT: $SSL_PORT
ports:
- "${SSL_PORT}:${SSL_PORT}"
If I do that
WARNING: The SSL_PORT variable is not set. Defaulting to a blank string.
ERROR: The Compose file '.\docker-compose.override.yml' is invalid because:
services.ssl.ports contains an invalid type, it should be a number, or an object
If I set it in the .env file, the container is built.
Is there any way I can set the value of SSL_PORT in docker-compose.override.yml AND use that same value?
There are only two ways to set value of SSL_PORT variable into docker-compose.override.yml:
1. Declaring default environment variables in an environment file named .env placed in the folder where the docker-compose command is executed.
2. Add SSL_PORT variable into environment variables when you execute docker-compose command. It would be either:
SSL_PORT=443 docker-compose -f docker-compose.override.yml up
or
export SSL_PORT=443
docker-compose -f docker-compose.override.yml up