I am currently using EKS private cluster with a public API server endpoint in order to use Azure DevOps environments(with Kubernetes service connection).
I have a requirement to make everything private in EKS.
Once EKS becomes private, it breaks everything in Azure DevOps as it is not able to reach the API server.
Any suggestion on how to communicate private kubernetes API server with azure devops would be appreciated.
If you're trying to target the cluster for deployment, you need a self-hosted agent that has a network route to your cluster.
The other capabilities exposed by the environment feature of Azure DevOps (i.e. monitoring the state of the cluster via the environment view) will not work -- they require a public-facing Kubernetes API to work.
If you don't mind the additional cost, VPN can be used to establish connection to the private EKS cluster.
Related
I setup a EKS cluster and integrated AWS Secrets Manager in it following the steps mentioned in https://github.com/aws/secrets-store-csi-driver-provider-aws and it worked as expected.
Now we have a requirement to integrate the AWS Secrets Manager on an on-premises k8s cluster and I am unable to follow the same steps as they seem to be explicitly for AWS EKS based clusters.
I googled around a bit and found you can call the Secrets Manager programmatically using one of the ways in https://docs.aws.amazon.com/secretsmanager/latest/userguide/asm_access.html, but this approach wont work for us.
Is there a k8s way to directly connect to AWS secrets Manager without setting up AWS-CLI and the OIDC cluster ID on the on-premises cluster?
Any help would be highly appreciated.
You can setup external OIDC providers with AWS and also setup K8s to with OIDC, but that is a lot of work.
AWS recently announced IAM Roles Anywhere which will let you use host based certificates to authenticate, but you will still have to call the Secrets Manager APIs.
If you are willing to retrieve secrets through etcd (which may store the secrets base64 encoded on the cluster) you can look at using the opensource External Secrets solution.
Creating a service connection to access non-private AKS cluster is straight forward, however if i want to create service connection for private AKS cluster is it possible from Azure Devops?
You can create New Kubernetes service connection using the KubeConfig option and click the dropdown arrow to choose Save without Verification
Also see Deploying to Private AKS Cluster
Please use below link
https://techcommunity.microsoft.com/t5/fasttrack-for-azure/using-azure-devops-to-deploy-an-application-on-aks-private/ba-p/2029630
I have impleted this solution in my place, we had private aks , we where unable to make service connection from azure devops to azure kubeneted,
we created a self hosted linux agent in the subnet where kubenetes is and add used my agent to run build and release pipeline
In the context of Azure Kubernetes Service (AKS), I would like to deploy some pods to a region not currently supported by Azure (in my case, Mexico). Is it possible to provision a non-Azure VM here in Mexico and attach it as a worker node to my AKS cluster?
Just to be clear, I want Azure to host the Kubernetes control plane. I want to spin out some Azure VMs within various supported regions. Then configure a non-Azure VM hosted in Mexico as a Kubernetes Node and attach it to the cluster.
(Soon there will be a Microsoft Azure Datacenter in Mexico and this problem will be moot. In the mean time, was hoping to monkey wrench it.)
You can't have a node pool with VMs that are not managed by Azure with AKS. You'll need to run your own k8s cluster if you want to do something like this. The closest you can get to something managed in Azure like AKS is to build your own Azure Arc enabled Kubernetes Cluster, but you'll need some skills with tools like Rancher, Kubespray, Kubeadm or something else.
I would like to connect to AKS API from a script on Azure VM inorder to scrape some metrics, check some stats, etc of the cluster.
Is there any approach (like an Azure policy or role and attaching it to VM) other than creating an user in azure AD or a service account in the AKS with clusterRole bound to it and referencing the certs/tokens from VM?
Thank you
To deploy services via Azure Devops to my kubernetes cluster, I need to create a Kubernetes Service Connection manually. I want to automate this by creating the service connection dynamically in Azure DevOps so I can delete and recreate the cluster and deployment. Is this possible? How can I do this?
you can create the service endpoint using the azure devops api,
check this out for api detail
this might be related