Kubebuilder: resource deletion not having expected side effects - kubernetes

Question

I'm creating a custom resource definition (CRD) with an associated controller using kubebuilder. My controller reconcile loop creates a deployment sub-resource and parents it to the custom resource using controllerutil.SetControllerReference(&myResource, deployment, r.Scheme). I've also configured my reconciler so "own" the sub-resource, as follows:
// SetupWithManager sets up the controller with the Manager.
func (r *MyResourceReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&mygroupv1alpha1.MyResource{}).
Owns(&appsv1.Deployment{}).
Complete(r)
}
However, when I run my controller locally using make run, I noticed that deleting the my CR (the root object) doesn't cause the Deployment sub-resource to get garbage collected. I also noticed that deleting the Deployment sub-resource doesn't trigger my reconciler to run. Why is this? Is there something I'm not doing or is this possibly a limitation of local development/testing?

Answer 1

Using #coderanger's hint, I could see that the metadata.ownerReferences weren't being set correctly when running the following command:
kubectl get deployments sample-deployment -o yaml
The problem was my controller's reconcile code. I was calling controllerutil.SetControllerReference(&myResource, deployment, r.Scheme) only after I'd already created and persisted the Deployment.
Buggy code
log.Info("Creating a deployment")
deployment := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: deploymentName,
Namespace: myResource.Namespace,
},
Spec: deploymentSpec,
}
if err = r.Create(ctx, deployment); err != nil {
log.Error(err, "Failed to create deployment")
if errors.IsInvalid(err) {
// Don't retry on validation errors
err = nil
}
return ctrl.Result{}, err
}
// Establish the parent-child relationship between my resource and the deployment
log.Info("Making my resource a parent of the deployment")
if err = controllerutil.SetControllerReference(&myResource, deployment, r.Scheme); err != nil {
log.Error(err, "Failed to set deployment controller reference")
return ctrl.Result{}, err
}
To fix it, I needed to swap the order of the call to r.Create and controllerutil.SetControllerReference:
Working code
log.Info("Creating a deployment")
deployment := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: deploymentName,
Namespace: myResource.Namespace,
},
Spec: deploymentSpec,
}
// Establish the parent-child relationship between my resource and the deployment
log.Info("Making my resource a parent of the deployment")
if err = controllerutil.SetControllerReference(&myResource, deployment, r.Scheme); err != nil {
log.Error(err, "Failed to set deployment controller reference")
return ctrl.Result{}, err
}
// Create the deployment with the parent/child relationship configured
if err = r.Create(ctx, deployment); err != nil {
log.Error(err, "Failed to create deployment")
if errors.IsInvalid(err) {
// Don't retry on validation errors
err = nil
}
return ctrl.Result{}, err
}
I was able to confirm that this worked by looking at the metadata.ownerReferences YAML data for my created deployment (using the command referenced above).
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: "2021-08-02T16:22:04Z"
generation: 1
name: sample-deployment
namespace: default
ownerReferences:
- apiVersion: resources.mydomain.io/v1alpha1
blockOwnerDeletion: true
controller: true
kind: MyResource
name: myresource-sample
uid: 6ebb146c-afc7-4601-bd75-58efc29beac9
resourceVersion: "569913"
uid: d9a4496f-7418-4831-ab87-4804dcd1f8aa