Pre-deploying Kubernetes loadbalancer with terraform on DigitalOcean? - kubernetes

Question

I'm learning about creating a k8s cluster on DO using terraform, I've been trying to take the ID of the single K8s node I've created, and reference it from the loadbalancer.
The main reasoning for this is so that I can declare the FQDN in the .tf file.
First, here is the cluster declaration:
variable "digitalocean_token" {}
provider "digitalocean" {
token = "${var.digitalocean_token}"
}
resource "digitalocean_kubernetes_cluster" "foo" {
name = "foo"
region = "nyc1"
version = "1.12.1-do.2"
node_pool {
name = "woker-pool"
size = "s-1vcpu-2gb"
node_count = 1
}
}
And here is the load balancer declaration:
resource "digitalocean_loadbalancer" "foo" {
name = "k8s-lb.nyc1"
region = "nyc1"
forwarding_rule {
entry_port = 80
entry_protocol = "http"
target_port = 80
target_protocol = "http"
}
droplet_ids = ["${digitalocean_kubernetes_cluster.foo.node_pool.0.id}"]
}
output "loadbalancer_ip" {
value = "${digitalocean_loadbalancer.foo.ip}"
}
resource "digitalocean_record" "terraform" {
domain = "example.com" # "${digitalocean_domain.example.name}"
type = "A"
name = "terraform"
value = "${digitalocean_loadbalancer.foo.ip}"
}
# Output the FQDN for the record
output "fqdn" {
value = "${digitalocean_record.terraform.fqdn}"
}
I'm guessing that maybe the digitalocean_loadbalancer resources is only setup to work with individual droplets?
Here are the output errors: when I run terraform apply:
* output.loadbalancer_ip: Resource 'digitalocean_loadbalancer.foo' not found for variable 'digitalocean_loadbalancer.foo.ip'
* digitalocean_record.terraform: Resource 'digitalocean_loadbalancer.foo' not found for variable 'digitalocean_loadbalancer.foo.ip'
* digitalocean_loadbalancer.foo: droplet_ids.0: cannot parse '' as int: strconv.ParseInt: parsing "d4292e64-9c0a-4afb-83fc-83f239bcb4af": invalid syntax
Pt. 2
I added a digitalocean_droplet resource, to see what kind of id was passed to the load balancer.
resource "digitalocean_droplet" "web" {
name = "web-1"
size = "s-1vcpu-1gb"
image = "ubuntu-18-04-x64"
region = "nyc1"
}
digitalocean_kubernetes_cluster.foo.node_pool.0.id = '6ae6a787-d837-4e78-a915-cb52155f66fe'
digitalocean_droplet.web.id = 132533158

Answer 1

So, the digitalocean_loadbalancer resource has an optional droplet_tag argument, which can be used to supply a common tag given to the created nodes/droplets.
However, when declaring a load-balancer inside kubernetes, a new one will still be created. So for now at least, it would appear that defining the domain/CNAME record with terraform isn't possible on digitalocean

Answer 2

You're using the wrong attribute reference for your load balancer droplet ids.
droplet_ids = ["${digitalocean_kubernetes_cluster.foo.node_pool.0.id}"]
This will use the node_pool id linked here
What you actually need to do is use the node_pool nodes id, which is referenced here
droplet_ids = "${digitalocean_kubernetes_cluster.foo.node_pool.0.nodes}"
The next problem you're going to have is that this returns a list of maps, and you'll need to build a list of ids from that. I'm not currently sure how to solve that, I'm afraid, but this should move you along hopefully.
It seems from your answer however, that what you want to do is update DNS for your loadbalancer.
You can do this external-dns using the digitalocean provider
Simply deploy this as a pod, specifying the required configuration, and ensure that the arg --source=service is set.
If you want to go a step further, and allow updating DNS with specific hostname, deploy an ingress controller like nginx-ingress and specify ingresses for your applications. The external-dns deployment (if you set --source=ingress) will the hostname from your ingress and update DNS for you.