I have a GCP cluster that was running on v1.21 and I have upgraded it to v1.22. There were some deprecated API calls that I was doing and I have managed to get rid of most of them but one.
$ kubent
2:02PM INF >>> Kube No Trouble `kubent` <<<
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.22 <<<
------------------------------------------------------------------------------------------
KIND NAMESPACE NAME API_VERSION REPLACE_WITH (SINCE)
Ingress elastic kibana-kibana networking.k8s.io/v1beta1 networking.k8s.io/v1 (1.19.0)
__________________________________________________________________________________________
I am pretty sure I have found where this is defined in my terraform scripts and I have tried to upgrade it following the release notes but to no avail.
resource "kubernetes_manifest" "nginx_frontend_config" {
manifest = {
"apiVersion" = "networking.gke.io/v1beta1"
"kind" = "FrontendConfig"
"metadata" = {
"name" = "nginx-frontend-config"
"namespace" = kubernetes_namespace.nginx.metadata[0].name
}
"spec" = {
"redirectToHttps" = {
"enabled" = true
"responseCodeName" = "FOUND"
}
"sslPolicy" = google_compute_ssl_policy.default.name
}
}
}
I have tried to upgrade the apiVersion from .../v1beta1 to .../v1 but I get the following error when running the terraform scripts:
╷
│ Error: Failed to determine GroupVersionResource for manifest
│
│ with kubernetes_manifest.nginx_frontend_config,
│ on nginx.tf line 83, in resource "kubernetes_manifest" "nginx_frontend_config":
│ 83: resource "kubernetes_manifest" "nginx_frontend_config" {
│
│ cannot select exact GV from REST mapper
╵
I have looked everywhere and I couldn't find a resource on how to define a Http -> Https redirect in Kubernetes v1.22. The official guides on GCP are referencing only the v1beta1 version. And in the Ingress Migration Guide it states to use the v1 version but that doesn't work for me.
P.S. I have also tried networking.k8s.io/v1 but it comes back with no matches for kind "FrontendConfig" in group "networking.k8s.io" when I run the terraform scripts.
How do I define a FrontendConfig for a redirect post v1.22?
Related
I am deploying azure service bus using terraform and yaml conf file. I am creating
azure service bus name pace, network rules for the service bus and service authorization rule for the name space using terraform.but,i want to define the multiple topics and multiple subscriptions under the topics in a yaml file which will be accessed by terraform as parameters during creating the resources "topic" and "subscription". I have defined the Multiple subscriptions as list value of the topic. The topics are created successfully but the Multiple subscriptions are not. The error , yaml, and terraform conf are given below.
Error: Incorrect attribute value type
│
│ on main.tf line 215, in resource "azurerm_servicebus_subscription" "subscription":
│ 215: name = each.value.servicebus_subscription
│ ├────────────────
│ │ each.value.servicebus_subscription is tuple with 2 elements
│
│ Inappropriate value for attribute "name": string required.
con.yaml
#-------
servicebus:
- servicebus_topic: tesTopic1
#enable_partitioning: "true"
servicebus_subscription: ['test-service1', 'test-service1']
- servicebus_topic: testTopic2
servicebus_subscription:['test-db1', 'test-service2']
pub_sub.tf
resource "azurerm_servicebus_subscription" "subscription" {
for_each = { for subscriptions in local.service_bus_conf : subscriptions.servicebus_topic=> subscriptions}
depends_on = [azurerm_servicebus_topic.topic]
name = each.value.servicebus_subscription
topic_id = data.azurerm_servicebus_topic.topic[each.value.servicebus_topic].id
}
````
How to deploy on K8 via Pulumi using the ArgoCD Helm Chart?
Pulumi up Diagnostics:
kubernetes:helm.sh/v3:Release (argocd):
error: failed to create chart from template: chart requires kubeVersion: >=1.22.0-0 which is incompatible with Kubernetes v1.20.0
THE CLUSTER VERSION IS: v1.23.0 verified on AWS. And NOT 1.20.0
ArgoCD install yaml used with CRD2Pulumi: https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/core-install.yaml
Source:
...
cluster = eks.Cluster("argo-example") # version="1.23"
# Cluster provider
provider = k8s.Provider(
"eks",
kubeconfig=cluster.kubeconfig.apply(lambda k: json.dumps(k))
#kubeconfig=cluster.kubeconfig
)
ns = k8s.core.v1.Namespace(
'argocd',
metadata={
"name": "argocd",
},
opts=pulumi.ResourceOptions(
provider=provider
)
)
argo = k8s.helm.v3.Release(
"argocd",
args=k8s.helm.v3.ReleaseArgs(
chart="argo-cd",
namespace=ns.metadata.name,
repository_opts=k8s.helm.v3.RepositoryOptsArgs(
repo="https://argoproj.github.io/argo-helm"
),
values={
"server": {
"service": {
"type": "LoadBalancer",
}
}
},
),
opts=pulumi.ResourceOptions(provider=provider, parent=ns),
)
Any ideas as to fixing this oddity between the version error and the actual cluster version?
I've tried:
Deleting everything and starting over.
Updating to the latest ArgoCD install yaml.
I could reproduce your issue, though I am not quite sure what causes the mismatch between versions. Better open an issue at pulumi's k8s repository.
Looking at the history of https://github.com/argoproj/argo-helm/blame/main/charts/argo-cd/Chart.yaml, you can see that the kubeversion requirement has been added after 5.9.1. So using that version successfully deploys the helm chart. E.g.
import * as k8s from "#pulumi/kubernetes";
const namespaceName = "argo";
const namespace = new k8s.core.v1.Namespace("namespace", {
metadata: {
name: namespaceName,
}
});
const argo = new k8s.helm.v3.Release("argo", {
repositoryOpts: {
repo: "https://argoproj.github.io/argo-helm"
},
chart: "argo-cd",
version: "5.9.1",
namespace: namespace.metadata.name,
})
(Not Recommended) Alternatively, you could also clone the source code of the chart, comment out the kubeVersion requirement in Chart.yaml and install the chart from your local path.
Upgrade helm. I had a similar issue where my k8s was 1.25 but helm complained it was 1.20. Tried everything else, upgrading helm worked.
I'm having the following issue.
I'm trying to deploy an EKS cluster with EKS addons (vpc cni, kubeproxy) and k8s addons (autoscaler, fluentbit). My ADO repo that has the .tf files is connected to TF Cloud, meaning my state is remote. I've recently found out that k8s/terraform won't let you deploy an EKS cluster and its addons in the same run, for some reason (I would get many random errors, at random times). I had to have a separate terraform apply for eks and addons, respectively.
So, I've decided to modularize my code.
Before, my main folder looked like this:
├── Deployment
│ └── main.tf
│ └── eks.tf
│ └── addons.tf
Now, my folder looks like this:
└───Deployment
│ │
│ └─── eks_deploy
│ │ main.tf
│ │ eks.tf
│ │
│ └─── addons_deploy
│ │ main.tf
│ │ addons.tf
And so, I initialize the same remote backend in both. So far, so good. Went ahead with a terraform apply in my eks_deploy folder. Deployed without problems, a clean EKS cluster with no addons. Now, it was time to deploy addons.
And that's where we have a problem.
My main.tf files are the exact same in both folders. And the file looks like this:
terraform {
backend "remote" {}
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 3.66.0"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.7.1"
}
helm = {
source = "hashicorp/helm"
version = ">= 2.4.1"
}
}
}
data "aws_eks_cluster" "cluster" {
name = module.eks-ssp.eks_cluster_id
}
data "aws_eks_cluster_auth" "cluster" {
name = module.eks-ssp.eks_cluster_id
}
# I am aware you're not supposed to hardcode your creds
provider "aws" {
access_key = "xxx"
secret_key = "xxx"
region = "xxx"
assume_role {
role_arn = "xxx"
}
}
provider "kubernetes" {
host = data.aws_eks_cluster.cluster.endpoint
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
token = data.aws_eks_cluster_auth.cluster.token
}
provider "helm" {
kubernetes {
host = data.aws_eks_cluster.cluster.endpoint
token = data.aws_eks_cluster_auth.cluster.token
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
}
}
The EKS cluster deployed without problems because it had an eks.tf file that contains the module and all needed info to deploy a cluster. However, my addon deployment throws the following errors:
╷
│ Error: Reference to undeclared module
│
│ on addons.tf line 60, in module "eks-ssp-kubernetes-addons":
│ 60: depends_on = [module.eks-ssp.self_managed_node_groups]
│
│ No module call named "eks-ssp" is declared in the root module.
╵
╷
│ Error: Reference to undeclared module
│
│ on main.tf line 22, in data "aws_eks_cluster" "cluster":
│ 22: name = module.eks-ssp.eks_cluster_id
│
│ No module call named "eks-ssp" is declared in the root module.
╵
╷
│ Error: Reference to undeclared module
│
│ on main.tf line 26, in data "aws_eks_cluster_auth" "cluster":
│ 26: name = module.eks-ssp.eks_cluster_id
│
│ No module call named "eks-ssp" is declared in the root module.
This is completely understandable, since the EKS cluster DOES NOT exist in the addon deployment, thus the addon deployment has no clue where to actually deploy those addons.
So my question is... how do I perform 2 different applies, for what's essentially the same resource (EKS), with each deployment being fully aware of each other (working as if they were in the same file and deployment? People mentioned "terragrunt", but I still don't understand how I could use it in my case, so if that's the solution you propose as well, please give a description of its way of use. There is also the following question - how would I connect the same repo, with 2 different folders/deployments, having separate applies? Does TF cloud even allow such a thing? At this point, I'm starting to think that a completely separate workspace, and hardcoded EKS values inside addons.tf is the only way. Thank you.
Terraform apply error 'The number of path segments is not divisible by 2' for Azure App Feature Flag
Why am I seeing this error? Hard to find any answer to this anywhere. I am using Terraform v2.93.0
and I also tried 2.90.0 and 2.56.0, and got the same problem. I was adding configs just fine but
as soon as I tried to configure a Feature Flag, it breaks the Terraform project AND
I am forced to rebuild re-init from scratch. Terraform is not able to recover on its own if I remove the config and running plan again.
╷
│ Error: while parsing resource ID: while parsing resource ID:
| The number of path segments is not divisible by 2 in
| "subscriptions/{key}/resourceGroups/my-config-test/providers/Microsoft.AppConfiguration/configurationStores/my-app-configuration/AppConfigurationFeature/.appconfig.featureflag/DEBUG/Label/my-functions-test"
│
│ while parsing resource ID: while parsing resource ID:
| The number of path segments is not divisible by 2 in
│ "subscriptions/{key}/resourceGroups/my-config-test/providers/Microsoft.AppConfiguration/configurationStores/my-app-configuration/AppConfigurationFeature/.appconfig.featureflag/DEBUG/Label/my-functions-test"
╵
╷
│ Error: obtaining auth token for "https://my-app-configuration.azconfig.io": getting authorization token for endpoint https://my-app-configuration.azconfig.io:
| obtaining Authorization Token from the Azure CLI: parsing json result from the Azure CLI: waiting for the Azure CLI: exit status 1: ERROR: The command failed with an unexpected error. Here is the traceback:
│ ERROR: [Errno 2] No such file or directory
WHY is the slash missing from the front of the ID????
And here is the config that breaks it:
resource "azurerm_app_configuration_feature" "my_functions_test_DEBUG" {
configuration_store_id = azurerm_app_configuration.my_app_configuration.id
description = "Debug Flag"
name = "DEBUG"
label = "my-functions-test"
enabled = false
}
When it is healthy, the apply on configs works, and looks like this:
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions in workspace "my-app-config-test"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
azurerm_resource_group.my_config_rg_test: Creating...
azurerm_resource_group.my_config_rg_test: Creation complete after 0s [id=/subscriptions/{key}/resourceGroups/my-config-test]
Ok, I figured it out. There is a bug: when create a azurerm_app_configuration_key resource, the key can be like so key = "/application/config.EXSTREAM_DOMAIN" BUT when you create a azurerm_app_configuration_feature, you will HOSE your terraform config if you try to set the name field to name = .appconfig.featureflag/DEBUG. Instead, just set the name field to DEBUG. If you don't do that, you have to completely reset your terraform and re-initialize all the resources. Had to learn the hard way. There error message was not helpful but could be updated to be helpful in this respect.
I'm running Kubernetes on GCP and doing changes via Terraform v0.11.14
When running terraform plan I'm getting the error messages here
Error: Error refreshing state: 2 errors occurred:
* module.cls-xxx-us-central1-a-dev.helm_release.cert-manager: 1 error occurred:
* module.cls-xxx-us-central1-a-dev.helm_release.cert-manager: helm_release.cert-manager: error installing: the server could not find the requested resource
* module.cls-xxx-us-central1-a-dev.helm_release.nginx: 1 error occurred:
* module.cls-xxx-us-central1-a-dev.helm_release.nginx: helm_release.nginx: error installing: the server could not find the requested resource
Here's a copy of my helm.tf
resource "helm_release" "nginx" {
depends_on = ["google_container_node_pool.tally-np"]
name = "ingress-nginx"
chart = "ingress-nginx/ingress-nginx"
namespace = "kube-system"
}
resource "helm_release" "cert-manager" {
depends_on = ["google_container_node_pool.tally-np"]
name = "cert-manager"
chart = "stable/cert-manager"
namespace = "kube-system"
set {
name = "ingressShim.defaultIssuerName"
value = "letsencrypt-production"
}
set {
name = "ingressShim.defaultIssuerKind"
value = "ClusterIssuer"
}
provisioner "local-exec" {
command = "gcloud container clusters get-credentials ${var.cluster_name} --zone ${google_container_cluster.cluster.zone} && kubectl create -f ${path.module}/letsencrypt-prod.yaml"
}
}
I've read that Helm deprecated most of the old chart repos so I tried adding the repositories and installing the charts locally under the namespace kube-system but so far the issue is still persisting.
Here's the list of versions for Terraform and it's providers
Terraform v0.11.14
provider.google v2.17.0
provider.helm v0.10.2
provider.kubernetes v1.9.0
provider.random v2.2.1
As the community is moving towards Helm v3, the maintainers have depreciated the old helm model where we had a single mono repo called stable. The new model is like each product having its own repo. On November 13, 2020 the stable and incubator charts repository reached the end of development and became archives.
The archived charts are now hosted at a new URL. To continue using the archived charts, you will have to make some tweaks in your helm workflow.
Sample workaround:
helm repo add new-stable https://charts.helm.sh/stable
helm fetch new-stable/prometheus-operator