I'm creating a KeyVault with Bicep and I want to create a secret in the vault, but only when there's no secret yet with the given name.
Checking if the KeyVault exists wasn't working, so I'm checking now if certain tag exists.
When creating the Vault I write a tag in the resource group. Afterwards I change the secret's password in the script and I run the script again, expecting the old password isn't overwritten. Unfortunately the secret is being recreated with the new password.
Any idea how to do a condition in Bicep, based on the existence of certain resource?
resource keyvault 'Microsoft.KeyVault/vaults#2019-09-01' = {
name: name
...
}
var rgWithDefaultTag = {
tags: {
keyVaultSecretName: ''
}
}
// Only create a new secret when a new KeyVault is created.
resource secret 'Microsoft.KeyVault/vaults/secrets#2021-04-01-preview' = if (empty(union(rgWithDefaultTag, resourceGroup()).tags['keyVaultSecretName'])) {
name: '${keyvault.name}/MySecret'
properties: {
value: 'value'
}
}
resource tag 'Microsoft.Resources/tags#2021-01-01' = {
name: 'default'
properties: {
tags: {
keyVaultSecretName: secret.name
}
}
}
Use this instead. It checks if the given tag exists on the resource group.
resource secret 'Microsoft.KeyVault/vaults/secrets#2019-09-01' = if (!contains(resourceGroup()).tags, 'keyVaultSecretName')) {
...
Related
I've an Azure Key Vault(KV) that has shared secrets and a cert that needs to be pulled into different deployments.
E.g. DEV, TEST, UAT, Production all have their own key vaults BUT need access to the shared KV for wild card ssl cert.
I've tried a number of approaches but each has errors. I'm doing something similar for KV within the deployment resource group without issues
Is it possible to have this and then use it as a module? Something like this...
sharedKV.bicep
var kvResourceGroup = 'project-shared-rg'
var subscriptionId = subscription().id
var name = 'project-shared-kv'
resource project_shared_kv 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
name: name
scope: resourceGroup(subscriptionId, kvResourceGroup )
}
And then uses like:
template.bicep
module shared_kv './sharedKeyVault/template.bicep' = {
name: 'sharedKeyVault'
}
resource add_secrect 'Microsoft.KeyVault/vaults/secrets#2021-06-01-preview' = {
name: '${shared_kv.name}/mySecretKey'
properties: {
contentType: 'string'
value: 'secretValue'
attributes: {
enabled: true
}
}
}
If you need to target a different resourceGroup (and/or sub) than the rest of the deployment, the module's scope property needs to target that RG/sub. e.g.
module shared_kv './sharedKeyVault/template.bicep' = {
scope: resourceGroup(kvSubscription, kvResourceGroupName)
name: 'sharedKeyVault'
params: {
subId: kvSubscription
rg: kvResourceGroupName
...
}
}
Ideally, the sub/rg for the KV would be passed in to the module rather than hardcoded (which you probably knew, but just in case...)
I want to do the following by using bicep:
Create a keyvault
Create a keyvault secret
Use this secret as the input for the creation of a Synapse Workspace(admin password)
I am using modules for creating all of the resources.
module keyVault 'modules/keyVault.bicep' = {
scope: resourceGroup
name: 'keyVault'
params: {
keyVaultName: keyVaultName
location: location
tenantID: subscription().tenantId
}
}
module keyVaultSecret 'modules/keyVaultSecret.bicep' = {
scope: resourceGroup
name: 'keyVaultSecretSynapseSQLAdminPassword'
params: {
secretName: 'synapseSQLAdministratorLoginPassword'
secretValue: synapseSqlAdministratorLoginPassword
keyVaultName: keyVaultName
keyVaultSecretName: '${keyVault.name}/synapseSQLAdministratorLoginPassword'
}
}
module synapse 'modules/synapseWs.bicep' = {
scope: resourceGroup
name: 'synapse'
params: {
synapseWSName: synapseWSName
synapseWSLocation: location
defaultAccountUrl: storageAccount.outputs.accURL
synapseSqlAdministratorLogin:synapseSqlAdministratorLogin
synapseSqlAdministratorLoginPassword: keyVault.getSecret('keyVaultSecretSynapseSQLAdminPassword')
managedResourceGroupName: '${environmentName}-cargo-${applicationName}-synapsemanaged-rg'
sqlPoolName: sqlPoolName
synapsePrivateLinkHubName: synapsePrivateLinkHubName
synapsePrivateLinkHubLocation: location
}
}
The getSecret function used in the line
synapseSqlAdministratorLoginPassword: keyVault.getSecret('keyVaultSecretSynapseSQLAdminPassword')
gives the error: "The type "module" does not contain function "getSecret"."
Apparently this function can only be used in resources. How could I do this in a different way?
Thanks
You has to reference the keyvault as existing in the bicep template. You can not use that function referencing a module. You has to reference the resource.
Create the keyvault with the module
Reference existing keyvault (as you just created)
Use the function on the existing keyvault reference.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/resource-declaration?tabs=azure-powershell#reference-existing-resources
I'm trying to add secrets to a task definition but can't find a way to specify which key to use from the key/value in the secret.
secrets = {
"DBUSER": ecs.Secret.from_secrets_manager(
sm.Secret.from_secret_complete_arn(
self, 'secret-dbuser',
'arn:aws:secretsmanager:eu-west-1:accountid:secret:secret-name').secret_value_from_json('DBUSER')
)
}
container: ecs.ContainerDefinition = task_definition.add_container(
"reports",
image=ecs.RepositoryImage.from_ecr_repository(
ecr.Repository.from_repository_name(self, "container", "container"), tag=image_tag,
),
memory_limit_mib=2048, logging=ecs.LogDriver.aws_logs(stream_prefix="container-"),
secrets=secrets
)
secret_value_from_json returns a SecretValue which isn't what I need.
I've also tried using from_secret_manager with filed='DBUSER' but that gives me an error like this
Invalid request provided: Create TaskDefinition: The Systems Manager parameter name specifie
d for secret DBUSER is invalid. The parameter name can be up to 2048 characters and include the following letters and symbols: a
-zA-Z0-9_.-, (Service: AmazonECS; Status Code: 400; Error Code: ClientException; Request ID
If the secret is in the same account/region, you should be able to do:
secrets = {
"DBUSER": ecs.Secret.from_secrets_manager(
# import the secret by its name
sm.Secret.from_secret_name_v2(self, 'secret-dbuser', '<secret-name-here>'),
# specify the specific field
'DBUSER'
)
}
container: ecs.ContainerDefinition = task_definition.add_container(
"reports",
image=ecs.RepositoryImage.from_ecr_repository(
ecr.Repository.from_repository_name(self, "container", "container"), tag=image_tag,
),
memory_limit_mib=2048, logging=ecs.LogDriver.aws_logs(stream_prefix="container-"),
secrets=secrets
)
ecs.Secret.from_secrets_manager() expects an ISecret and a field.
See also https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_ecs/Secret.html#aws_cdk.aws_ecs.Secret.from_secrets_manager
Using kubectl we can create docker registry authentication secret as follows
kubectl create secret docker-registry regsecret \
--docker-server=docker.example.com \
--docker-username=kube \
--docker-password=PW_STRING \
--docker-email=my#email.com \
How do i create this secret using terraform, i saw this link, it has data, in the flow of terraform the kubernetes instance is being created in azure and i get the data required from there and i created something like below
resource "kubernetes_secret" "docker-registry" {
metadata {
name = "registry-credentials"
}
data = {
docker-server = data.azurerm_container_registry.docker_registry_data.login_server
docker-username = data.azurerm_container_registry.docker_registry_data.admin_username
docker-password = data.azurerm_container_registry.docker_registry_data.admin_password
}
}
It seems that it is wrong as the images are not being pulled. What am i missing here.
If you run following command
kubectl create secret docker-registry regsecret \
--docker-server=docker.example.com \
--docker-username=kube \
--docker-password=PW_STRING \
--docker-email=my#email.com
It will create a secret like following
$ kubectl get secrets regsecret -o yaml
apiVersion: v1
data:
.dockerconfigjson: eyJhdXRocyI6eyJkb2NrZXIuZXhhbXBsZS5jb20iOnsidXNlcm5hbWUiOiJrdWJlIiwicGFzc3dvcmQiOiJQV19TVFJJTkciLCJlbWFpbCI6Im15QGVtYWlsLmNvbSIsImF1dGgiOiJhM1ZpWlRwUVYxOVRWRkpKVGtjPSJ9fX0=
kind: Secret
metadata:
creationTimestamp: "2020-06-01T18:31:07Z"
name: regsecret
namespace: default
resourceVersion: "42304"
selfLink: /api/v1/namespaces/default/secrets/regsecret
uid: 59054483-2789-4dd2-9321-74d911eef610
type: kubernetes.io/dockerconfigjson
If we decode .dockerconfigjson we will get
{"auths":{"docker.example.com":{"username":"kube","password":"PW_STRING","email":"my#email.com","auth":"a3ViZTpQV19TVFJJTkc="}}}
So, how can we do that using terraform?
I created a file config.json with following data
{"auths":{"${docker-server}":{"username":"${docker-username}","password":"${docker-password}","email":"${docker-email}","auth":"${auth}"}}}
Then in main.tf file
resource "kubernetes_secret" "docker-registry" {
metadata {
name = "regsecret"
}
data = {
".dockerconfigjson" = "${data.template_file.docker_config_script.rendered}"
}
type = "kubernetes.io/dockerconfigjson"
}
data "template_file" "docker_config_script" {
template = "${file("${path.module}/config.json")}"
vars = {
docker-username = "${var.docker-username}"
docker-password = "${var.docker-password}"
docker-server = "${var.docker-server}"
docker-email = "${var.docker-email}"
auth = base64encode("${var.docker-username}:${var.docker-password}")
}
}
then run
$ terraform apply
This will generate same secrets. Hope it will helps
I would suggest creating a azurerm_role_assignement to give aks access to the acr:
resource "azurerm_role_assignment" "aks_sp_acr" {
scope = azurerm_container_registry.acr.id
role_definition_name = "AcrPull"
principal_id = var.service_principal_obj_id
depends_on = [
azurerm_kubernetes_cluster.aks,
azurerm_container_registry.acr
]
}
Update
You can create the service principal in the azure portal or with az cli and use client_id, client_secret and object-id in terraform.
Get Client_id and Object_id by running az ad sp list --filter "displayName eq '<name>'". The secret has to be created in the Certificates & secrets tab of the service principal. See this guide: https://pixelrobots.co.uk/2018/11/first-look-at-terraform-and-the-azure-cloud-shell/
Just set all three as variable, eg for obj_id:
variable "service_principal_obj_id" {
default = "<object-id>"
}
Now use the credentials with aks:
resource "azurerm_kubernetes_cluster" "aks" {
...
service_principal {
client_id = var.service_principal_app_id
client_secret = var.service_principal_password
}
...
}
And set the object id in the acr as described above.
Alternative
You can create the service principal with terraform (only works if you have the necessary permissions). https://www.terraform.io/docs/providers/azuread/r/service_principal.html combined with a random_password resource:
resource "azuread_application" "aks_sp" {
name = "somename"
available_to_other_tenants = false
oauth2_allow_implicit_flow = false
}
resource "azuread_service_principal" "aks_sp" {
application_id = azuread_application.aks_sp.application_id
depends_on = [
azuread_application.aks_sp
]
}
resource "azuread_service_principal_password" "aks_sp_pwd" {
service_principal_id = azuread_service_principal.aks_sp.id
value = random_password.aks_sp_pwd.result
end_date = "2099-01-01T01:02:03Z"
depends_on = [
azuread_service_principal.aks_sp
]
}
You need to assign the role "Conributer" to the sp and can use it directly in aks / acr.
resource "azurerm_role_assignment" "aks_sp_role_assignment" {
scope = var.subscription_id
role_definition_name = "Contributor"
principal_id = azuread_service_principal.aks_sp.id
depends_on = [
azuread_service_principal_password.aks_sp_pwd
]
}
Use them with aks:
resource "azurerm_kubernetes_cluster" "aks" {
...
service_principal {
client_id = azuread_service_principal.aks_sp.app_id
client_secret = azuread_service_principal_password.aks_sp_pwd.value
}
...
}
and the role assignment:
resource "azurerm_role_assignment" "aks_sp_acr" {
scope = azurerm_container_registry.acr.id
role_definition_name = "AcrPull"
principal_id = azuread_service_principal.aks_sp.object_id
depends_on = [
azurerm_kubernetes_cluster.aks,
azurerm_container_registry.acr
]
}
Update secret example
resource "random_password" "aks_sp_pwd" {
length = 32
special = true
}
I'm using the terraform kubernetes-provider and I'd like to translate something like this kubectl command into TF:
kubectl create secret generic my-secret --from-file mysecret.json
It seems, however the secret resource's data field expects only a TF map.
I've tried something like
data "template_file" "my-secret" {
template = "${file("${path.module}/my-secret.json")}"
}
resource "kubernetes_secret" "sgw-config" {
metadata {
name = "my-secret"
}
type = "Opaque"
data = "{data.template_file.my-secret.template}"
}
But it complains that this is not a map. So, I can do something like this:
data = {
"my-secret.json" = "{data.template_file.my-secret.template}"
}
But this will write the secret with a top-level field named my-secret.json and when I volume mount it, it won't work with other resources.
What is the trick here?
as long the file is UTF-8 encoded you can use something like this
resource "kubernetes_secret" "some-secret" {
metadata {
name = "some-secret"
namespace = kubernetes_namespace.some-ns.metadata.0.name
labels = {
"sensitive" = "true"
"app" = "my-app"
}
}
data = {
"file.txt" = file("${path.cwd}/your/relative/path/to/file.txt")
}
}
If the file is a binary one you will have an error like
Call to function "file" failed: contents of
/your/relative/path/to/file.txt are not valid UTF-8; use the
filebase64 function to obtain the Base64 encoded contents or the other
file functions (e.g. filemd5, filesha256) to obtain file hashing
results instead.
I tried encoding the file in base64 but then the problem is that the resulting text will be re-encoded in base64 by the provider. So I guess there is no solution for binary files at the moment...
I'll edit with what I find next for binaries.
This might be a bit off-topic, but I've been facing similar problem except that the file might not be present in which case the terraform [plan|apply] fails.
To be exact: I needed to duplicate a secret from one namespace to another one.
I realized that by using hashicorp/external provider.
The steps are pretty simple:
Load data by calling external program
Refer to the data in kubernetes_secret resource
The program should accept (and process) JSON on STDIN and produce valid JSON on STDOUT as response to the parameters passed-in in the STDIN's JSON.
Example shell script:
#!/bin/bash
set -e
/bin/echo -n '{ "token": "'
kubectl get -n consul secrets/hashicorp-consul-bootstrap-acl-token --template={{.data.token}}
/bin/echo -n '"}'
tarraform source:
data "external" "token" {
program = ["sh", "${path.module}/consul-token.sh"]
}
resource "kubernetes_secret" "consul-token" {
depends_on = [data.external.token]
metadata {
name = "consul-token"
namespace = "app"
}
data = {
token = base64decode(data.external.token.result.token)
}
}
and requirements:
terraform {
required_providers {
external = {
source = "hashicorp/external"
version = ">= 2.0.0"
}
}
}
Just use
https://www.terraform.io/docs/providers/kubernetes/r/config_map.html#binary_data
resource "kubernetes_config_map" "example" {
metadata {
name = "my-config"
}
binary_data = {
"my_payload.bin" = "${filebase64("${path.module}/my_payload.bin")}"
}
}
I believe you can use binary_data attribute in the secret now.
e.g.
binary_data = {
"my_payload.bin" = "${filebase64("${path.module}/my_payload.bin")}"
}
reference:
https://github.com/hashicorp/terraform-provider-kubernetes/pull/1228
https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret#binary_data
Basically you need to provide a map like this :
resource "kubernetes_secret" "sgw-config" {
metadata {
name = "my-secret"
}
type = "Opaque"
data {
"key1" = "value1"
"key2" = "value2"
}
}
you can refer to your internal variables using
resource "kubernetes_secret" "sgw-config" {
metadata {
name = "my-secret"
}
type = "Opaque"
data {
"USERNAME" = "${var.some_variable}"
"PASSWORD" = "${random_string.root_password.result}"
}
}
It seems if you run the command kubectl create secret generic my-secret --from-file mysecret.json
and then
$ kubectl get secrets my-secret -o yaml
apiVersion: v1
data:
my-secret.json: ewogICA.....
kind: Secret
metadata:
creationTimestamp: "2019-03-25T18:20:43Z"
name: my-secret
namespace: default
resourceVersion: "67026"
selfLink: /api/v1/namespaces/default/secrets/my-secret
uid: b397a29c-4f2a-11e9-9806-000c290425d0
type: Opaque
it stores it similarly with the filename as the single key. When I mount this in a volume/volumeMount it works as expected. I was afraid that it wouldn't but when I create the secret using the --from-file argument, this is exactly how it stores it.