ALB Ingress Controller, Terminating TLS via AWS ACM. How to go End-To-End? - kubernetes

Question

We currently use the AWS ALB Ingress Controller to front out ingress, and terminate SSL using a certificate from AWS ACM. This work fine.
Is there a way, to also encrypt the traffic from the Load balancer to the cluster?
Here is what I attempted
Install/Configure Cert Manager
Add a TLS Secret to the Ingress
Change the ingress annotation to set the backend protocol to HTTPS alb.ingress.kubernetes.io/backend-protocol: HTTPS
This... results in a 502 gateway error
here is my current, working ingress, with only the relevant parts still shown.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
alb.ingress.kubernetes.io/backend-protocol: HTTP
alb.ingress.kubernetes.io/certificate-arn: real-cert-arn
alb.ingress.kubernetes.io/group.name: public.monitor
alb.ingress.kubernetes.io/group.order: "40"
alb.ingress.kubernetes.io/healthcheck-path: /
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
alb.ingress.kubernetes.io/load-balancer-name: monitoring-public-qa
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10
cert-manager.io/cluster-issuer: cert-manager-r53-qa
kubernetes.io/ingress.class: alb
spec:
rules:
- host: goldilocks.qa.realdomain.com
http:
paths:
- backend:
service:
name: goldilocks-dashboard
port:
name: http
path: /*
pathType: ImplementationSpecific
tls:
- hosts:
- goldilocks.qa.realdomain.com
secretName: goldilocks-qa-cert
status:
loadBalancer:
ingress:
- hostname: real-lb-address.us-gov-west-1.elb.amazonaws.com
my (staging) cert exists, and appears fine.
[ec2-user#ip-10-17-2-102 ~]$ kubectl get cert -n goldilocks goldilocks-qa-cert -o yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: goldilocks-qa-cert
namespace: goldilocks
ownerReferences:
- apiVersion: networking.k8s.io/v1
blockOwnerDeletion: true
controller: true
kind: Ingress
name: goldilocks-dashboard
uid: 2494e5be-e624-471b-afd3-c8d56f5dc853
resourceVersion: "81003934"
uid: d494a1ec-7509-474e-b154-d5db8ceb86c0
spec:
dnsNames:
- goldilocks.qa.realdomain.com
issuerRef:
group: cert-manager.io
kind: ClusterIssuer
name: cert-manager-r53-qa
secretName: goldilocks-qa-cert
usages:
- digital signature
- key encipherment
status:
conditions:
- lastTransitionTime: "2023-02-08T16:06:56Z"
message: Certificate is up to date and has not expired
The fact I can not figure out what to google to find this answer, leads me to think that I'm attempting to do something weird? I understand I could just terminate the TLS on the pod, but I didn't want to rely on the lets encrypt to provide me good/valid certs, I just want the traffic encrypted.