How to replace .env file placeholders with deployment variables in bitbucket pipeline - deployment

I have a bitbucket repository.
I have Deployment environment variables in bitbucket:
BITBUCKET_VARIABLE_PORT : 8080
In my bitbucket-pipelines.yaml script I can Write the variable into the .env file like this:
<...>
step: &deploy-to-environment
name: Deploy to environment
deployment: environment
caches:
- node
script:
- echo Create .env file
- echo "PORT=$BITBUCKET_VARIABLE_PORT" > .env
- cat .env
<...>
But I would like to avoid rewriting the whole lines of .env file.
Is it possible to implement the following?
I would like to have .env file with placeholders (.env file content):
<...>
PORT=<BITBUCKET_VARIABLE_PORT>
HOST=<BITBUCKET_VARIABLE_HOST>
<...>
Replace these placeholders in yaml script section:
<...>
step: &deploy-to-environment
name: Deploy to environment
deployment: environment
caches:
- node
script:
- <replace_placeholders_here_in_script>
<...>

You can use sed to replace string in .env file
sed -i "s/BITBUCKET_VARIABLE_PORT/...xxxx.../" .env
sed -i "s/BITBUCKET_VARIABLE_HOST/...hostname.../" .env
You can use also variables for replacement
MYPORT=XXX
sed -i "s/BITBUCKET_VARIABLE_PORT/$MYPORT/" .env

I had the same issue lately and didn't want to include 1 by 1 the env variable, what I ended up doing:
#1 Create in repository a .env.example file (This file will hold variable that required for this repository without values or default values that are not sensitive)
APP_NAME=MyApp
APP_ENV=
APP_KEY=
APP_DEBUG=true
Then I created a pipeline.sh
echo "[+] Building enviroment variables"
# Get contents of example file
ENV_CONTENT=$(cat ./.env.example)
# Output the content into sh script
echo "#! /bin/bash
echo \"
${ENV_CONTENT}
\"" > ./env.sh
# sed replace as key=${BITBUCKET_ENV_VARIABLE:-default_value_from_example}
sed -i -E "s/^([A-Z_]+)=(.*)$/\1=\${\1:-\2}/g" ./env.sh
chmod +x ./build/env.sh
# Exec the env sh script and output content to .env file
./build/env.sh > ./build/.env
So in case you have defined in .env.example APP_NAME and also defined it in bitbucket Repository Variables the final .env will have key APP_NAME=value_of_bitbucket_variable

Here is the way I found to do it, you just have to be careful with those echo commands (here I'm making env for firebase but you can swap your bitbucket variables in how you like)
definitions:
steps:
- step: &Env
name: Create .env file
trigger: manual
script:
- echo VITE_FIREBASE_API_KEY="${VITE_FIREBASE_API_KEY}" >> .env
- echo VITE_FIREBASE_AUTH_DOMAIN="${VITE_FIREBASE_AUTH_DOMAIN}" >> .env
- echo VITE_FIREBASE_PROJECT_ID="${VITE_FIREBASE_PROJECT_ID}" >> .env
- echo VITE_FIREBASE_STORAGE_BUCKET="${VITE_FIREBASE_STORAGE_BUCKET}" >> .env
- echo VITE_FIREBASE_MESSAGING_SENDER_ID="${VITE_FIREBASE_MESSAGING_SENDER_ID}" >> .env
- echo VITE_FIREBASE_APP_ID="${VITE_FIREBASE_APP_ID}" >> .env
- more .env
artifacts:
- .env
pipelines:
default:
- step: *Env

Related

Using external action output in outer git action step

I have this git action for my build
...
- name: Building S3 Instance
uses: charlie87041/s3-actions#main
id: s3
env:
AWS_S3_BUCKET: 'xxx'
AWS_ACCESS_KEY_ID: 'xxx'
AWS_SECRET_ACCESS_KEY: 'xxxxx'
AWS_REGION: 'xxx'
- name: Updating EC2 [Develop] instance
uses: appleboy/ssh-action#master
with:
host: ${{secrets.EC2HOST}}
key: ${{secrets.EC2KEY}}
username: xxx
envs: TESTING
script: |
cd ~/devdir
export BUCKET_USER=${{steps.s3.outputs.user_id}}
export BUCKET_USER_KEY=${{steps.s3.outputs.user_key}}
docker login
docker-compose down --remove-orphans
docker system prune -a -f
docker pull yyyy
docker-compose up -d
And this is the important function in charlie87041/s3-actions#main
generate_keys () {
RSP=$(aws iam create-access-key --user-name $USER);
BUCKET_ACCESS_ID=$(echo $RSP | jq -r '.AccessKey.AccessKeyId');
BUCKET_ACCESS_KEY=$(echo $RSP | jq -r '.AccessKey.SecretAccessKey');
echo "user_id=$BUCKET_ACCESS_ID" >> $GITHUB_OUTPUT
echo "user_key=$BUCKET_ACCESS_KEY" >> $GITHUB_OUTPUT
echo "::set-output name=BUCKET_ACCESS_KEY::$BUCKET_ACCESS_KEY"
echo "::set-output name=BUCKET_ACCESS_ID::$BUCKET_ACCESS_ID"
}
I need to update env variables in container with BUCKET_USER and BUCKET_USER_KEY, but these always return null when echo the container. How do I do this?
Not that set-output was deprecated recently (oct. 2022)
If you are using self-hosted runners make sure they are updated to version 2.297.0 or greater.
If you are using runner on github.com directly, you would need to change
echo "::set-output name=BUCKET_ACCESS_KEY::$BUCKET_ACCESS_KEY"
with
echo "BUCKET_ACCESS_KEY=$BUCKET_ACCESS_KEY" >> $GITHUB_OUTPUT
I am not sure an export within the script would work.
Using with directives, as in issue 154 might be more effective
with:
BUCKET_USER: ${{steps.s3.outputs.user_id}}
...
script: |
...

How to mask a environment set in GitHub Workflow

I am using the following to set a env variable:
- name: get root token
run: |
echo "ROOT_TOKEN=$(some command | base64 --decode)" >> $GITHUB_ENV
Then I use it in another run within a Python script:
- name: init
run: |
python3 scripts/create_entries.py
Actually this works great, but the value of ROOT_TOKEN is printed in the Workflow console:
Run python3 scripts/create_entries.py
python3 scripts/create_entries.py
shell: /usr/bin/bash -e ***0***
env:
DATA: ***
CONFIG: /home/debian/runner/_work/_temp/config_1646400032032
ROOT_TOKEN: <this is shown>
I tried to mask it using ::add-mask:: like this (but unfortunately that does not work):
- name: get root token
run: |
echo "::add-mask::ROOT_TOKEN=$(some command | base64 --decode)" >> $GITHUB_ENV
Does anyone know how to mask the value of ROOT_TOKEN in the Workflow console?

Docker-compose .env file internal variable substitution

I am trying to perform variable substitution inside of a .env file but have not had any luck so far.
I've been looking though the docker-compose documentation and have not found anything mentioning this (or any examples online) but it seems like something that would be surprising if not possible.
What I am talking about is doing something like this in my .env file:
// .env
SOME_LOCATION=/path/to/some/location
CONFIG_FILE=${SOME_LOCATION}/config
CONSTANT_FILE=${SOME_LOCATION}/constants
(This example makes CONFIG_FILE equal to the string $${SOME_LOCATION}/config and same thing happens with CONSTANT_FILE)
I realize that this is possible inside of the compose.yml file with syntax like this but can it be done just inside the .env file?
I'm using docker-compose version 1.24.1 if it's not possible then I will just copy past these kinds of things but it always feels dirty copying the same values through your configuration.
You can change or add .env variable with export command in shell scripting.
File path like this;
-project
-> docker-compose.yml
-> .env
-> start.sh
docker-compose.yml
services:
jira:
image: atlassian/jira-software
volumes:
- ${CONFIG_FILE}:/var/atlassian/application-data/jira
ports:
- 8080:8080
volumes:
jira_vol:
external: false
start.sh
export "CONFIG_FILE=${SOME_LOCATION}/jira_vol"
docker-compose up -d
.env
SOME_LOCATION=./volumes
Finally run this command;
sh start.sh
start.sh file will add CONFIG_FILE variable and run docker compose. And you can use CONFIG_FILE and SOME_LOCATION in docker-compose.yml.

Can .env file make environment variables?

I tried to use a .env file to make environment variable, but it doesn't work.
These are my steps:
version "3"
services:
web:
image: php-fpm:5.6.30
env_file:
- .env
This is .env file
TEST_ENV="HELLO WORLD"
It doesn't work when I start the container:
var_dump(getenv("TEST_ENV")); // output NULL
For me it seems to work. Maybe this can help you:
├── docker-compose.yaml
├── .env
└── myphp
├── Dockerfile
└── script.php
My .env file
TEST_ENV="HELLO WORLD"
My docker-compose.yaml:
version: "3"
services:
web:
build: ./myphp
env_file: .env
So my docker-compose.yaml will build the image myphp. The dockerfile looks like this:
FROM php:5.6.30-fpm
COPY script.php /var/script.php
My script.php
<?php
var_dump(getenv('TEST_ENV'));
exit;
Than I perform docker-compose up -d --build. This will build my image and add the php script in it and it will run a container instance of that image.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
15f0289bfbe8 test_web "docker-php-entryp..." 3 seconds ago Up 1 second 9000/tcp test_web_1
I'm accessing my container
$ docker exec -it 15f0289bfbe8 bash
And I'm going the the /var folder where I've put my script (check dockerfile) and I'm exexcuting it + also just printing env var:
root#15f0289bfbe8:/var/www/html# cd /var/
root#15f0289bfbe8:/var# ls
backups cache lib local lock log mail opt run script.php spool tmp www
root#15f0289bfbe8:/var# php -f script.php
string(13) ""HELLO WORLD""
root#15f0289bfbe8:/var# echo $TEST_ENV
"HELLO WORLD"
root#15f0289bfbe8:/var#

Call Perl Script using Ansible

I have the below .sh code which need to get converted to Ansible tasks.
#!/bin/sh
echo "Installing Sonar"
SONAR_HOME=/tui/hybris/sonar
if [ ! -d "$SONAR_HOME" ]; then
mkdir -p $SONAR_HOME
fi
cd $SONAR_HOME
wget https://s3-eu-west-1.amazonaws.com/tuiuk/source/sonarqube/sonarqube-5.4.zip
unzip sonarqube-5.4.zip
echo "Modifying Sonar config file"
cd sonarqube-5.4/conf
perl -p -i -e 's/#sonar.jdbc.username=/sonar.jdbc.username=sonar/g' sonar.properties
perl -p -i -e 's/#sonar.jdbc.password=/sonar.jdbc.password=sonar/g' sonar.properties
perl -p -i -e 's/#sonar.jdbc.url=jdbc:mysql/sonar.jdbc.url=jdbc:mysql/g' sonar.properties
cd $SONAR_HOME
echo "downloading and copying plugins"
wget https://s3-eu-west-1.amazonaws.com/tuiuk/source/sonarqube/sonarqube5.4_plugins.zip
unzip sonarqube5.4_plugins.zip
cp plugins/* sonarqube-5.4/extensions/plugins/
cd sonarqube-5.4/bin/linux-x86-64
echo "Starting Sonar"
./sonar.sh start
Below is my task.I got stuck where I need to execute perl script. Could any of you help me in proceeding further.
- hosts: docker_test
tasks:
- name: Creates directory
file: path=/tui/hybris/sonar state=directory mode=0777
sudo: yes
- name: Installing Sonar
get_url:
url: "https://s3-eu-west-1.amazonaws.com/tuiuk/source/sonarqube/sonarqube-5.4.zip"
dest: "/tui/hybris/sonar/sonarqube-5.4.zip"
register: get_solr
- debug:
msg: "solr was downloaded"
when: get_solr|changed
- name: Unzip SonarQube
unarchive: src=/tui/hybris/sonar/sonarqube-5.4.zip dest=/tui/hybris/sonar copy=no
I bet you don't need perl here, use lineinfile with regex option (if you need to modify a single line in the file) or replace module (if you need to modify all occurrences).
Just call perl with command or shell-module:
- task: Modifying Sonar config file
shell: cd sonarqube-5.4/conf && perl -p -i -e ...