For instance, can I write something like pulumi up -ec2_count=15 so that ec2_count is my custom arg I interpret in my Pulumi code?
I couldn't find such capability in their docs, but it's strange that it's not possible...
You can dynamically pass config entries to to pulumi up by using the -c flag:
pulumi up -c APPLICATION_DOCKER_IMAGE_VERSION=latest
and require it in your code
Pulumi.run(ctx -> {
var config = ctx.config();
var version = config.require("APPLICATION_DOCKER_IMAGE_VERSION");
.
.
.
});
I think that it could be possible, but maybe is not the best approach to use Pulumi.
You can get this from environment variables or config set with Pulumi cli.
Ex:
Env Vars:
os.Getenv("ec2_count")
Config:
pulumi config set dev:myvars:ec2_count: 1
config.require("ec2_count")
https://www.pulumi.com/docs/intro/concepts/config/
Related
Context:
I'm reusing terraform modules and I deploy microservices using helm provider within terraform.
Problem:
I'm trying to translate this line into terraform code, to get the current image tag live from prod (in the interest of reusing it). I'm already using kubernetes provider's auth and doesn't make sense to pull kubectl in my CI just for this.
k get deploy my-deployment -n staging -o jsonpath='{$.spec.template.spec.containers[:1].image}'
Kubernetes terraform provider doesn't seem to support data blocks nor helm provider outputs blocks.
Does anyone know how could we get (read) the image tag of a deployment using terraform?
EDIT:
My deployment looks like this:
resource "helm_release" "example" {
name = "my-redis-release"
repository = "https://charts.bitnami.com/bitnami"
chart = "redis"
version = "6.0.1"
values = [
"${file("values.yaml")}"
]
set {
name = "image.tag"
value = "latest"
}
}
The tag will be a hash that will change often and passed on from another repo.
latest in this case should be replaced by the current running tag in the cluster. I can get it using kubectl, using the line above, but not sure how using terraform.
It turns out there are multiple ways of doing it, where the easiest one for me is to reference the set argument of the helm_release resource:
output "helm_image_tag" {
value = [ for setting in helm_release.example.set : setting.value if setting.name == "image.tag" ]
}
The output will then be a list where you can reference it in a shell script (or another scripting language):
+ helm_image_tag = [
+ "latest",
]
If the list format does not suit you, you can create a map output:
output "helm_image_tag" {
value = { for setting in helm_release.example.set : setting.name => setting.value if setting.name == "image.tag" }
}
This produces the following output:
+ helm_image_tag = {
+ "image.tag" = "latest"
}
By using terraform output helm_image_tag you can access this output value and decide what to do with it in the CI.
I have a build config file that looks like:
steps:
...
<i use the ${_IMAGE} variable around 4 times here>
...
images: ['${_IMAGE}']
options:
dynamic_substitutions: true
substitutions:
_IMAGE: http://example.com/image-${_ENVIRON}
And I trigger the build like:
gcloud builds submit . --config=config.yaml --substitutions=_ENVIRON=prod
What I expected is for the gcloud to substitute the _ENVIRON variable in my script and then substitute the _IMAGE variable so that it'd expand to 'http://example.com/image-prod' - but instead I'm getting the following error:
ERROR: (gcloud.builds.submit) INVALID_ARGUMENT: generic::invalid_argument: key "_ENVIRON" in the substitution data is not matched in the template
What can I do to make that work? I really want to be able to change the environment easily with a sub and without the need to change anything in code
As you've seen, this isn't possible.
If the only use of _ENVIRON is by _IMAGE, why not drop the substitions from config.yaml and use _IMAGE as the substitution:
ENVIRON="prod"
IMAGE: http://example.com/image-${ENVIRON}
gcloud builds submit . \
--config=config.yaml \
--substitutions=_IMAGE=${IMAGE}
I have a downloaded file through helm inspect called sftp.yaml
I have a parameter in that sftp.yaml file:-
sftp:
allowedMACs: "hmac-sha2-512"
allowedCiphers: aes256-ctr
Now if i install the corresponding helm chart after commenting out the entire line of "allowedMACs" from custom values files i.e. "sftp.yaml", then K8s takes the delta of sftp.yaml and the actual values.yaml and then use values.yaml's "allowedMACs".
However What i want is if "allowedMACs" line is commented in "sftp.yaml" custom values file, then it should not set the env variable at all, or sets it as null.
presently my deployment file's env section looks like
- name: MACs
value: {{ default "" .Values.sftp.allowedMACs | quote }}
You need to either override (with new value) or unset the value, if you only comment out the section you are not doing any of the above and the default value is going to be used.
Basically you are looking to unset a default value. As per banzaicloud example this can be done like so:
helm install stable/chart-name --set sftp.allowedMACs=null
You can also use override value file in a similar way:
sftp:
allowedMACs: null
allowedCiphers: aes256-ctr
This is available in Helm since version 2.6. If you like in-depth information you can review the issue and the subsequent PR that introduced the feature.
yeah I think helm would retrieve values from all values files, so if allowedMACs is in one of those it'll get populated. If this parameter is affected only by sftp.yaml file should it really belong only to it and would i make sense to remove it from main values.yaml?
I am trying to pass an environment variable in my deployment that should define a prefix based on a version number:
env:
- name: INDEX_PREFIX
value: myapp-$(VERSION)
$(VERSION) is not defined in my deployment but is set in the docker image used by the pod.
I tried to use both $() and ${} but VERSION is not interpolated in the environment of my pod. In my pod shell doing export TEST=myapp-${VERSION} does work though.
Is there any way to achieve what I am looking for? ie setting an environment variable in my deployment that reference an environment variable set in the docker image?
VERSION is an environment variable of the docker image. So you can assign it a value either inside the container or by passing
env:
- name : VERSION
value : YOUR-VALUE
In your case, VERSION is either set by a script inside the docker container or in the Dockerfile.
You can do :
In the Dockerfile, adding ENV INDEX_PREFIX myapp-${VERSION}
Adding a script to your entrypoint as
export INDEX_PREFIX=myapp-${VERSION}
In case you can't modify Dockerfile, you can try to :
Get the image entrypoint file from the docker image (ie: /IMAGE-entrypoint.sh) and the image args(ie: IMAGE-ARGS). you can use docker inspect IMAGE.
Override the container command and args in the pod spec using a script.
command:
- '/bin/sh'
args:
- '-c'
- |
set -e
set -x
export INDEX_PREFIX=myapp-${VERSION}
IMAGE-entrypoint.sh IMAGE-ARGS
k8s documentation : https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
Hope it could help you.
I'm using Terraform in a modular fashion in order to build out my infrastructure. I do this by having a configuration file that calls in the different modules. I want to pass an infrastructure variable which picks up what tagged version of the Github repository the application should be building out. Most importantly I'm trying to figure out how to make a concatenation of a string happen in the "source" variable of the configuration file.
module "athenaelb" {
source = "${concat("git::https://github.com/ORG/REPONAME.git?ref=",var.infra_version)}"
aws_access_key = "${var.aws_access_key}"
aws_secret_key = "${var.aws_secret_key}"
aws_region = "${var.aws_region}"
availability_zones = "${var.availability_zones}"
subnet_id = "${var.subnet_id}"
security_group = "${var.athenaelb_security_group}"
branch_name = "${var.branch_name}"
env = "${var.env}"
sns_topic = "${var.sns_topic}"
s3_bucket = "${var.elb_s3_bucket}"
athena_elb_sns_topic = "${var.athena_elb_sns_topic}"
infra_version = "${var.infra_version}"
}
I want it to compile and for the source to look like this (for example): git::https://github.com/ORG/REPONAME.git?ref=v1
Anyone have any thoughts on how to make this work?
Thanks,
Keren
This is not possible currently in Terraform itself.
The only way to achieve something like this is to use a separate script to interact with the git repository that Terraform clones into a subdirectory of the .terraform/modules directory and switch it to a different tag depending on which version you need. This is non-ideal since Terraform organizes these into directories based on a hash of the module path, but if you can identify the module in question it is safe to run git checkout within these repositories as long as you do not run terraform get again afterwards.
For more details and discussion on this issue, see issue #1439 in Terraform's issue tracker, where this feature was requested.
You could use envsubst or python jinja and use these wrapper scripts in your pipeline deploy script to actually build the scripts from .envsubst and .jinja files before your terraform plan/apply
https://github.com/uvoo/process-templates/tree/main/scripts
I wish terraform would support this but my guess is they never will so just add some simple functions/files into deploy scripts which is usually the best way to deploy.