I have a small application comprising three services:
A single page application (SPA) served from nginx
A simple nodejs HTTP API used by the SPA
An MQtt broker exposing ports 1883 and 9001
Ideally I'd like the all to be served from the same subdomain and static IP address and have been trying to configure this in Kubernetes on the Google Cloud Platform.
I've created deployments for each of the services, with the SPA exposing port 80, the API 3000 and the MQTT broker 1883/9001. I've then followed the instructions here to set up a static IP and a Service to route to the SPA, then created similar services for the API and the MQTT app. (I've initally adapted these from deployments and services generated from a docker-compose file and Kompose).
The SPA and API seem to work fine but the MQTT service does not. When I run kubetl get events I see:
Error creating load balancer (will retry): failed to ensure load balancer for service default/mqtt-broker: failed to create forwarding rule for load balancer (a5529f2a9bdaf11e8b35d42010a84005(default/mqtt-broker)): googleapi: Error 400: Invalid value for field 'resource.IPAddress': '35.190.221.113'. Specified IP address is in-use and would result in a conflict., invalid
So I'm wondering if I should be creating a single service to route to the three deployments but can't find any documentation or examples that explain how to do this for a non http service.
I guess I could put the mqtt service on a separate IP address but this seems to be hacking around the problem rather than solving it.
Thanks in advance for any advice.
I eventually found an almost identical use case to my own on this github repository.
In essence, they are creating the MQTT broker on a separate static IP and using Kubernetes API calls to expose the details to the front end, which they explain in the following comment at the top of the web.yaml file:
This needs a bit of trickery
as it needs to expose the LB ip address for the MQTT server. That
requires kubernetes API calls to look it up, and the ability to
store it somewhere (we put it in a secret). To be secure this is
done with a dedicated service account and an init container.
https://github.com/IBM/ny-power
Related
I have an app that is deployed in k8s cluster and the frontend and the backend of the app are exposed, is there a way to not expose the backend ? i thought about the api gateway is it going to fulfills my requests if yes how ? and if not what's the alternatives ??
Thank you in advance
I tried the gateway kong and it did not work out very well
Your question requires more detail for me to be certain, however I will take a stab at pointing you in the right direction.
Typically, most apps have a frontend which serves the HTML, and any static assets such as images, css and javascript (like a single page app such as ReactJS)
If you have some SPA, then you will likely have a backend API written in something like Node / Python / PHP / Java to serve your frontend app with dynamic data.
If your frontend and backend are exposed to the internet, this is OK and expected.
If your backend was not exposed to the internet, then it would be impossible for your frontend to load dynamic data.
That said, you mentioned that you are using or at least tried to use an API Gateway. Typically, you would not expose your backend directly to the internet. Rather, you would expose your API Gateway to the internet, with the API Gateway acting as a reverse proxy to your backend.
In order to achieve this (in Kubernetes), you would typically create a Service type Load Balancer for your API Gateway. And configure a Service type Cluster IP for your backend.
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default that is used if you don't explicitly specify a type for a Service. You can expose the service to the public with an Ingress or the Gateway API.
I understand that the question was asked and discussed in different formats before. However, I still miss clear guidelines on how to handle the situation.
Our keycloak setup has multiple keycloak replicas and is behind a load balancer without a fixed ip in a separate infrastructure. So that our DNS records look like:
CNAME keycloak.acme.com public-lb.acme.com
And public-lb.acme.com forwards the request to specific instances of keycloak.
One of our end-user applications is located in a completely different infrastructure with strict access. The end-user application is built using java and is using Keycloak integration org.keycloak:keycloak-servlet-filter-adapter. We do not have any custom adapters and simply follow "standard" configuration:
{
"auth-server-url" : "https://keycloak.acme.com",
..
However, this does not work since keycloak.acme.com ip address have to be whitelisted in that "special" infrastructure. So that validation requests from the application inside the "special" infrastructure do not hit the keycloak. And we cannot whitelist the ip, since the ip of our load balancer public-lb.acme.com is not fixed and changes with time.
We have a "tunnel" between the keycloak infrastructure and that "special" infrastructure with a dedicated ip cidr range which is whitelisted.
Hence we have create a special internal load balancer that is in the tunnels cidr range and forwards requests to the keycloak replicas. Unfortunately that internal load balancer does not have a fixed ip address, and can change within time.
Since we do not have fixed ip address, is the only correct method is to use add DNS record inside the "special" infrastructure pointing to the internal load balancer? Something like:
CNAME keycloak.acme.com internal-lb.acme.com
Or are there any alternative solutions? I understand the historical reasons behind this.
Cloud Run is providing a domain *.run.app to access the service deployed. I am wondering how Google Cloud Run handling requests from one to another Cloud Run service. Is all the service to service communication internal even we have a custom domain instead of *.run.app?
The definition of "internal" is not clear.
Your request stay in the Google Network. Is it internal or external?
To resolve the Custom Domain, a DNS resolution request (port 53) is performed on the public network, but the content of the request stays in the Google Network and forwarded after the resolution. Is it internal or external?
So, as long as you use Google Services (in premium network option), you don't go out of the Google Network and thus you can consider this as highly secured.
I assume, my answer isn't very clear, in fact all depend if you trust or not the Google Cloud network.
My backend and frontend is deployed on Google Kubernetes Engine (GKE). And lets consider these steps -
User opens up browser and points to yyy.yyy.yyy:8080.
Browser loads the login page.
User enters username/password and presses "login" button.
Browser (ReactJS App) sends request to REST backend xxx.xxx.xxx:7070 to do the validation.
Now my question is, how can we inject the backend (in this case xxx.xxx.xxx) ip frontend pod? Because this ip is not static (i dont want to make it static), and will be provided by GKE. I don't want to proxy my backend call through frontend either. Neither I want to use nginx, the reason is, one more POD, and config to maintain. Just don't want to have nginx for only this reason.
Should I create ingress? I mean something like-
`/` <-- serves ReactJS app
`/api` <--- serves REST api
Or there is a way to inject the ip of backend POD?
What is the best approach?
Thanks in advance.
You can't access the pods directly from the outside. The only way in is through a service of type LoadBalancer. External IP you get for such a service is static for GKE (AWS gives a DNS name instead)
For every service of type LoadBalancer the cloud provider will spin up a load balancer that you have to pay for. You can funnel all http traffic through a single ingress service that would fan out your requests based on host DNS or URI path and other options. Which is what pretty much everyone does. If you want to setup something like that google for ingress
Here is usecase:
I have two apps in Bluemix: app1 and app2
app1 is accessible through the internet using its route (e.g. app1.mybluemix.net)
app2 doesn't have any route to prevent from being accessible through the internet.
app2 may expose a REST API.
How do I call app2 from app1 inside Bluemix?
An example of communicating to an application without a route is implemented in this Microservice Shipping sample.
This is an EJB Liberty application that runs on Bluemix without a route and subscribes to the Bluemix MQ Light service. The sender of the messages is the Microservice Orders sample application, which binds to the same MQ Light service.
Going the REST API route will mean you must have an externally accessible route. However, you could secure it using keys and tokens.
It would be easier to use one of the services in Bluemix as an "RPC" layer between the two applications. You could use one of the queue services (MQLight, RabbitMQ) or Redis to pass messages between the applications to execute commands.
These service bindings are internal and won't be exposed externally unlike the REST API.
Alternatively, you could expose the REST API from App2 and use authentication to control access.
There are two ways you can prevent access.
Put your microservice inside a Bluemix Container and utilize private IPs https://new-console.ng.bluemix.net/docs/containers/container_security_network.html#container_cli_ips_byoip
Use API Connect as a API Gateway/proxy to the private IP being in your container microservice.
Use Bluemix Dedicated to deploy app2. Bluemix dedicated provides firewall capabilities and you could set it up so that it only accepts requests from app1's IP address.
Use Bluemix Local when it becomes available with the same approach where you use your corporate firewall to only accept requests that come from your App1 IP Address. This is an expensive alternative compared to a public PAAS.
Use the API Connect Service which replaced the API Management Service to:
Specify what users can access your apis
Specify the number of requests per day or other unit of time
Provides a API Gateway to securely call the other service App2.
I expect at some point a software network defined solution will be considered as part of the offering.