k8s: helm install ingress-nginx only create IngressClass? - kubernetes

Question

I'm setting up two ingresses in different namespaces with ingress-nginx (https://github.com/kubernetes/ingress-nginx). My understanding is that I need to install ingress-nginx for each namespace, which creates the IngressClass I need.
I've installed the ingress-nginx with this:
helm install ingress-ns1 ingress-nginx/ingress-nginx \
--namespace ns1 \
--set controller.ingressClassResource.name=ns1-class \
--set controller.scope.namespace=ns1 \
--set controller.ingressClassByName=true
then the same again for namespace ns2. My understanding this created the IngressClasses I need and seems to work.
I've also got an Ingress configuration templated by helm that uses the IngresClasses:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: "{{ .Values.ingress.name }}"
namespace: "{{ .Values.namespace }}"
annotations:
cert-manager.io/cluster-issuer: "{{ .Values.issuer.name }}"
nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
spec:
ingressClassName: {{ .Values.ingress.ingressClassName }}
rules:
{{- range $v := .Values.ingress.rules }}
- host: {{ $v.host.name }}
http:
paths:
{{- range $p := $v.host.paths }}
- path: {{ $p.path.path }}
pathType: Prefix
backend:
service:
name: {{ $p.path.name }}
port:
number: {{ $p.path.port }}
{{- end }}
{{- end }}
tls:
- hosts:
{{- range $v := .Values.ingress.rules }}
- {{ $v.host.name }}
{{- end }}
secretName: "{{ .Values.issuer.name }}"
This seems to work and uses the IngressClass which i've templated into {{ .Values.ingress.ingressClassName }}. These end up being ns1-class and ns2-class.
However. I then end up with 4 loadbalancers created, rather than two!
Looking at k9s, seems that installing the ingress-nginx with helm installs the two IngressClasses which I want, but also adds its own ingress controller pods. I only want the two created with my Ingress definition above.
How do I still setup the IngressClass to use ingress-nginx, but not have the controller created by installing ingress-nginx?
I've read this: (https://kubernetes.github.io/ingress-nginx/user-guide/multiple-ingress/#multiple-ingress-controllers) a few times, I find it quite confusing as there are snippets of configuration that I don't know what to do with/where to put.