NGINX Ingress - Redis Module add on - kubernetes

Question

Hi there I'm having some trouble adding in a 3rd party module into the Kubernetes NGINX-Ingress custom image, and haven't been successful in figuring out how to do it.
I have seen one other has had this problem, and they seemed to of compiled it from scratch but dont seem to provide details as to how they added the file.
I'm installing the ingress controller via Helm, with a simple values.yaml file to make the alterations shown below:
values.yaml
controller:
image:
registry: [registry]
repository: [repo]
image: [image]
tag: 1.0.2
pullPolicy: Always
config:
entries:
main-snippets: "load_module /etc/nginx/modules/ngx_http_redis2_module.so"
prometheus:
create: true
Which was accepted when doing a helm install, in pulling a custom image into the pod/container for the ingress controller. I have been able to dynamically compile the module into a .so file that I'm keeping locally at this time to add into the custom image. But the issue that I'm having it seems that when building a Docker file for this custom image I seem to be unable to add the module file.
Dockerfile
FROM nginx/nginx-ingress:2.1.0
#Adds nginx redis2 module
USER root
COPY ngx_http_redis2_module.so /etc/nginx/modules/ngx_http_redis2_module.so
USER www-data
The dockerfile is what I'm using above to attempt to add the file into the proper place, which should be under the /etc/nginx/modules folder like the others. But after running the pod, and bashing into it I'm only seeing the following:
-rwxr-xr-x 1 www-data www-data 164256 Sep 26 17:40 ngx_http_auth_digest_module.so
-rwxr-xr-x 1 www-data www-data 5388256 Sep 26 17:40 ngx_http_brotli_filter_module.so
-rwxr-xr-x 1 www-data www-data 78152 Sep 26 17:40 ngx_http_brotli_static_module.so
-rwxr-xr-x 1 www-data www-data 100704 Sep 26 17:40 ngx_http_geoip2_module.so
-rwxr-xr-x 1 www-data www-data 113056 Sep 26 17:40 ngx_http_influxdb_module.so
-rwxr-xr-x 1 www-data www-data 267080 Sep 26 17:40 ngx_http_modsecurity_module.so
-rwxr-xr-x 1 www-data www-data 2468184 Sep 26 17:40 ngx_http_opentracing_module.so
-rwxr-xr-x 1 www-data www-data 74856 Sep 26 17:40 ngx_stream_geoip2_module.so
It seems that I'm making the image incorrectly or something else entirely and any help would be appreciated.

Answer 1

So I found that I was doing a few things wrong partially, but thanks to #jordanm for helping me out here.
I was able to run it using the docker run -it --entrypoint=/bin/sh command pointing to my image. I saw that my commands, of adding the file on there was correct but I was doing a few things incorrectly. First off I was using the wrong image, should of been using the controller, so here is the proper dockerfile:
Dockerfile
FROM k8s.gcr.io/ingress-nginx/controller:v1.1.1
#Adds nginx redis2 module
USER root
COPY --chmod=744 ngx_http_redis2_module.so /etc/nginx/modules/ngx_http_redis2_module.so
USER www-data
Also I had to tweak my values.yml file for helm to be a bit different also:
values.yml
controller:
image:
registry: [registry]
image: [image]
tag: "1.0.5"
digest: "sha256:[sha value]"
pullPolicy: Always
config:
main-snippets: "load_module /etc/nginx/modules/ngx_http_redis2_module.so;"
prometheus:
create: true
I had to add in the digest value which didnt seem to be stated in the helm instructions. In order to get the SHA256 value use this command docker inspect --format='{{index .RepoDigests 0}}' [IMAGE]:[TAG] and use that output in the digest value.
What was happening which I didnt take notice, was the container was failing so fast and a different one was being spun up before I even noticed it. So what I did was I went onto my test control plane / node and uninstalled NGINX first, then I did a sudo docker system prune -af which removed unused images. This way I was being guaranteed my image was being pulled through and deployed, and not reverting back to a different image.
I dont know why but the description of the pod would state that it was deploying my image, but I believe under the hood it would use another image.