Failed to retrieve sa token using terraform - kubernetes

I need to retrieve SA token using output in my pipeline, i found an solution in here
Retrieve token data from Kubernetes Service Account in Terraform
but still dont work and get this error:
│ Error: Invalid function argument
│
│ on access.tf line 51, in output "deploy_user_token":
│ 51: value = lookup(data.kubernetes_secret.deploy_user_secret.data, "token")
│ ├────────────────
│ │ data.kubernetes_secret.deploy_user_secret.data has a sensitive value
│
│ Invalid value for "inputMap" parameter: argument must not be null.
My code:
resource "kubernetes_service_account" "deploy_user" {
depends_on = [kubernetes_namespace.namespace]
metadata {
name = "deploy-user"
namespace = var.namespace
}
}
resource "kubernetes_role" "deploy_user_full_access" {
metadata {
name = "deploy-user-full-access"
namespace = var.namespace
}
rule {
api_groups = ["", "extensions", "apps", "networking.istio.io"]
resources = ["*"]
verbs = ["*"]
}
rule {
api_groups = ["batch"]
resources = ["jobs", "cronjobs"]
verbs = ["*"]
}
}
resource "kubernetes_role_binding" "deploy_user_view" {
metadata {
name = "deploy-user-view"
namespace = var.namespace
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "Role"
name = kubernetes_role.deploy_user_full_access.metadata.0.name
}
subject {
kind = "ServiceAccount"
name = kubernetes_service_account.deploy_user.metadata.0.name
namespace = var.namespace
}
}
data "kubernetes_secret" "deploy_user_secret" {
metadata {
name = kubernetes_service_account.deploy_user.default_secret_name
}
}
output "deploy_user_token" {
value = lookup(data.kubernetes_secret.deploy_user_secret.data, "token")
}
someone have an idea that what i do wrong?
Thanks!

it seems that you missing the namespace declaration on your data object, you need it to look like that:
data "kubernetes_secret" "deploy_user_secret" {
metadata {
name = kubernetes_service_account.deploy_user.default_secret_name
namespace = var.namespace
}
}
you also need the set sensitive = true on your output:
output "deploy_user_token" {
sensitive = true
value = lookup(data.kubernetes_secret.deploy_user_secret.data, "token")
}

Related

Terraform: can't create `kubernetes_secret` in `google_container_cluster` because "secrets is forbidden"

I have build a kubernetes cluster on google cloud and I am now trying to use the kubernetes_secret resource to create a secret. Here is my configuration:
resource "google_service_account" "default" {
account_id = "gke-service-account"
display_name = "GKE Service Account"
}
resource "google_container_cluster" "cluster" {
name = "${var.cluster-name}-${terraform.workspace}"
location = var.region
initial_node_count = 1
project = var.project-id
remove_default_node_pool = true
}
resource "google_container_node_pool" "cluster_node_pool" {
name = "${var.cluster-name}-${terraform.workspace}-node-pool"
location = var.region
cluster = google_container_cluster.cluster.name
node_count = 1
node_config {
preemptible = true
machine_type = "e2-medium"
service_account = google_service_account.default.email
oauth_scopes = [
"https://www.googleapis.com/auth/cloud-platform"
]
}
}
provider "kubernetes" {
host = "https://${google_container_cluster.cluster.endpoint}"
client_certificate = base64decode(google_container_cluster.cluster.master_auth.0.client_certificate)
client_key = base64decode(google_container_cluster.cluster.master_auth.0.client_key)
cluster_ca_certificate = base64decode(google_container_cluster.cluster.master_auth.0.cluster_ca_certificate)
}
resource "kubernetes_secret" "cloudsql-credentials" {
metadata {
name = "database-credentials" # The name of the secret
}
data = {
connection-name = var.database-connection-name
username = var.database-user
password = var.database-password
}
type = "kubernetes.io/basic-auth"
However I get the following error when creating the kubernetes_secret resource:
╷
│ Error: secrets is forbidden: User "system:anonymous" cannot create resource "secrets" in API group "" in the namespace "default"
│
│ with module.kubernetes-cluster.kubernetes_secret.cloudsql-credentials,
│ on gke/main.tf line 58, in resource "kubernetes_secret" "cloudsql-credentials":
│ 58: resource "kubernetes_secret" "cloudsql-credentials" {
│
╵
What am I missing here? I really don't understand. In the documentation I have found the following that could maybe help:
Depending on whether you have a current context set this may require `config_context_auth_info` and/or `config_context_cluster` and/or `config_context`
But it is not clear at all how this should be set and there are no examples provided. Any help will be appreciated. Thank you.

how to use terraform output to deploy image inside kubernetes cluster in modular approch?

well, I'm new in terraforming and also in Kubernetes, I faced an issue in deploying images after creating a Kubernetes cluster.
I have created a module that creates a Kubernetes cluster and provides output for the Kube config data.
now I'm using the code below but i need to run terraform apply 2 times because first time the local file is not created and terraform could not connect to kubernetes or helm or kubectl but if i run the command twice it works as expected.
any solution?
Note: i also applied the solution as on the comment section of the
code and that comment did now work either.
File : main.tf
module "deploy_lke" {
source = "./modules/linode/kubernetes"
token = var.token
k8s_version = var.k8s_version
label = var.label
region = var.region
tags = var.tags
instance_type = var.instance_type
number_of_instance = var.number_of_instance
min = var.min
max = var.max
}
module "deploy_image" {
source = "./modules/kubernetes"
kube_config_path = module.deploy_lke.kubeconfig
dockerconfigjson = file("./secret/docker-sec.json")
deploy_name = var.deploy_name
desire_replicas = var.desire_replicas
image_link = var.image_link
image_name = var.image_name
image_port = var.image_port
ip_type = var.ip_type
max_replicas_val = var.max_replicas_val
min_replicas_val = var.min_replicas_val
service_name = var.service_name
}
File : ./module/linode/kubernetes
terraform {
required_providers {
linode = {
source = "linode/linode"
version = "1.29.4"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.0.0"
}
}
}
provider "linode" {
token = var.token
}
resource "linode_lke_cluster" "gaintplay-web-lke" {
k8s_version = var.k8s_version
label = var.label
region = var.region
tags = var.tags
pool {
type = var.instance_type
count = var.number_of_instance
autoscaler {
min = var.min
max = var.max
}
}
lifecycle {
ignore_changes = [
pool.0.count
]
}
}
output "kubeconfig" {
value = linode_lke_cluster.gaintplay-web-lke.kubeconfig
}
output "api_endpoints" {
value = linode_lke_cluster.gaintplay-web-lke.api_endpoints
}
File : ./module/kubernetes
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.0.0"
}
}
}
resource "local_file" "kube_config_file" {
content = var.kube_config_path
filename = "${path.module}/config"
}
provider "kubernetes" {
config_path = var.kube_config_path
}
provider "helm" {
kubernetes {
config_path = var.kube_config_path
}
}
resource "kubernetes_secret" "docker_secret" {
metadata {
name = "docker-cfg"
}
data = {
".dockerconfigjson" = var.dockerconfigjson
}
type = "kubernetes.io/dockerconfigjson"
}
resource "kubernetes_deployment" "beta" {
depends_on = [
kubernetes_secret.docker_secret
]
metadata {
name = var.deploy_name
namespace = "default"
}
spec {
replicas = var.desire_replicas
selector {
match_labels = {
app = var.deploy_name
}
}
template {
metadata {
labels = {
app = var.deploy_name
}
}
spec {
image_pull_secrets {
name = kubernetes_secret.docker_secret.metadata[0].name
}
container {
image_pull_policy = "Always"
image = var.image_link
name = var.image_name
port {
container_port = var.image_port
}
}
}
}
}
}
# provider "kubernetes" {
# host = "${yamldecode(var.kube_config_path).clusters.0.cluster.server}"
# client_certificate = "${base64decode(yamldecode(var.kube_config_path).users.0.user.client-certificate-data)}"
# client_key = "${base64decode(yamldecode(var.kube_config_path).users.0.user.client-key-data)}"
# cluster_ca_certificate = "${base64decode(yamldecode(var.kube_config_path).clusters.0.cluster.certificate-authority-data)}"
# }
# provider "helm" {
# kubernetes {
# host = "${yamldecode(var.kube_config_path).clusters.0.cluster.server}"
# client_certificate = "${base64decode(yamldecode(var.kube_config_path).users.0.user.client-certificate-data)}"
# client_key = "${base64decode(yamldecode(var.kube_config_path).users.0.user.client-key-data)}"
# cluster_ca_certificate = "${base64decode(yamldecode(var.kube_config_path).clusters.0.cluster.certificate-authority-data)}"
# }
# }
If i use the command as it is i got this error in terraform plan that the file is not found and i need to run it twice.
Invalid attribute in provider configuration
with module.deploy_image.provider["registry.terraform.io/hashicorp/kubernetes"],
on modules/kubernetes/main.tf line 13, in provider "kubernetes":
13: provider "kubernetes" {
'config_path' refers to an invalid path: "modules/kubernetes/config": stat modules/kubernetes/config: no such file or directory
and
If I use commented code i get error like this:
│ Error: Unsupported attribute
│
│ on main.tf line 35, in provider "kubernetes":
│ 35: host = "${yamldecode(linode_lke_cluster.gaintplay-web-lke.kubeconfig).clusters.0.cluster.server}"
│
│ Can't access attributes on a primitive-typed value (string).
╵
╷
│ Error: Unsupported attribute
│
│ on main.tf line 36, in provider "kubernetes":
│ 36: client_certificate = "${base64decode(yamldecode(linode_lke_cluster.gaintplay-web-lke.kubeconfig).users.0.user.client-certificate-data)}"
│
│ Can't access attributes on a primitive-typed value (string).

Error when adding tags to Snowflake resource (role)

I am using:
Terraform v1.2.9
on windows_amd64
provider registry.terraform.io/snowflake-labs/snowflake v0.42.1
My main.tf file is:
terraform {
required_version = ">= 1.1.7"
backend "http" {
}
required_providers {
snowflake = {
source = "Snowflake-Labs/snowflake"
version = "~> 0.42"
}
}
}
provider "snowflake" {
username = "xxxxxx"
account = "yyyyyy"
region = "canada-central.azure"
}
When I add the following tags to a Snowflake role, I have an error. Can you help?
resource "snowflake_role" "operations_manager" {
name = "OPERATIONS_MANAGER"
comment = "A comment"
tag = {
managed-with = "Terraform",
owner = "Joe Smith"
}
}
Error: Unsupported argument
│
│ on functional_roles.tf line 35, in resource "snowflake_role" "operations_manager":
│ 35: tag = {
│
│ An argument named "tag" is not expected here. Did you mean to define a block of type "tag"?

Terraform - kubernetes - create spec env-from if variable exixts

I try to create resources based on variables.
variable.tf
variable "apps" {
default = null
type = map(object({
name = string
type = string
secrets = optional(map(string))
}))
}
terraform.tfvars
apps = {
"myfirst" = {
name = "myfirst"
type = "deploy"
secrets = {
"FIRST_VAR" = "TestVariable",
"SECOND_VAR" = "SecontTestVariable",
"THIRD" = "NothingHere"
}
},
"second" ={
name = "second"
type = "deploy"
secrets = {
"SECRET_VAR" = "SecretVar"
}
},
"simlepod" ={
name = "simplepod"
type = "deploy"
},
"another" ={
name = "another"
type = "pod"
And my main.tf
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
version = "2.9.0"
}
}
experiments = [module_variable_optional_attrs]
}
provider "kubernetes" {
config_path = "~/.kube/config"
# Configuration options
}
resource "kubernetes_secret" "secret" {
for_each = { for k in compact([for k, v in var.apps: v.secrets != null ? k : ""]): k => var.apps[k] }
metadata {
name = "${each.value.name}-secret"
}
data = each.value["secrets"]
}
resource "kubernetes_pod" "test" {
for_each = { for k in compact([for k, v in var.apps: v.type =="deploy" ? k : ""]): k => var.apps[k] }
metadata {
name = "app-${each.value.name}"
}
spec {
container {
image = "nginx:1.21.6"
name = "test-${each.value.name}"
env_from {
secret_ref {
name = kubernetes_secret.secret[each.value.name].metadata[0].name
}
}
resources {
limits = {
cpu = "0.5"
memory = "512Mi"
}
requests = {
cpu = "250m"
memory = "50Mi"
}
}
}
}
timeouts {
create = "60s"
}
}
And this produces error because not all objects in apps have secret variable.
terraform plan
╷
│ Warning: Experimental feature "module_variable_optional_attrs" is active
│
│ on main.tf line 8, in terraform:
│ 8: experiments = [module_variable_optional_attrs]
│
│ Experimental features are subject to breaking changes in future minor or patch
│ releases, based on feedback.
│
│ If you have feedback on the design of this feature, please open a GitHub issue to
│ discuss it.
╵
╷
│ Error: Unsupported block type
│
│ on main.tf line 68, in resource "kubernetes_pod" "test":
│ 68: dynamic "secret" {
│
│ Blocks of type "secret" are not expected here.
How to use expression to create env_from only when object has secrets variable?
I've found solution
...
dynamic "env_from" {
for_each = each.value.secrets[*]
content {
secret_ref {
name = kubernetes_secret.secret[each.value.name].metadata[0].name
}
}
}
...
and seems to work

terraform - Unable to fetch service account from Kubernetes: serviceaccounts "<name of service account>" not found

This was working perfectly fine before but for some reason it no longer is, would appreciate if someone can help fix this:
My terraform code as follows, have replaced key info. with "<>" just for sharing publicly here:
Outer main.tf has this:
module "<name>_service_account" {
source = "../modules/kubernetes/service-account"
name = "<name>-deployer"
}
# Create <name> platform namespace
resource "kubernetes_namespace" "<name>-platform" {
metadata {
name = "<name>-platform"
}
}
The service account main.tf module:
resource "kubernetes_service_account" "serviceaccount" {
metadata {
name = var.name
namespace = "kube-system"
}
}
resource "kubernetes_cluster_role_binding" "serviceaccount" {
metadata {
name = var.name
}
subject {
kind = "User"
name = "system:serviceaccount:kube-system:${var.name}"
}
role_ref {
kind = "ClusterRole"
name = "cluster-admin"
api_group = "rbac.authorization.k8s.io"
}
}
data "kubernetes_service_account" "serviceaccount" {
metadata {
name = var.name
namespace = "kube-system"
}
depends_on = [
resource.kubernetes_service_account.serviceaccount
]
}
data "kubernetes_secret" "serviceaccount" {
metadata {
name = data.kubernetes_service_account.serviceaccount.default_secret_name
namespace = "kube-system"
}
binary_data = {
"token": ""
}
depends_on = [
resource.kubernetes_service_account.serviceaccount
]
}
My outputs.tf for the above module:
output "secret_token" {
sensitive = true
value = lookup(data.kubernetes_secret.serviceaccount.binary_data, "token")
}
The error that I get in my terraform pipeline:
│ Error: Unable to fetch service account from Kubernetes: serviceaccounts "<name>-deployer" not found
│
│ with module.<name>_service_account.data.kubernetes_service_account.serviceaccount,
│ on ../modules/kubernetes/service-account/main.tf line 27, in data "kubernetes_service_account" "serviceaccount":
│ 27: data "kubernetes_service_account" "serviceaccount" {
Figured it out, this is a new environment/project and I had the terraform refresh stage still in the pipeline hence why it couldnt find the service account, removing that and just letting the plan and apply run first solved it.