Recently I have tried to install openshift 4.2 on VMWare and followed this documentation https://blog.openshift.com/openshift-4-2-vsphere-install-with-static-ips/ so I was able to install it successfully and it's working fine. But this installation is using a single LoadBalancer (HAProxy) for everything.
So In my case, the IP of LoadBalancer was 10.68.33.62 then I mapped the URL like below
10.68.33.62 api.openshift4.example.com
10.68.33.62 api-int.openshift4.example.com
10.68.33.62 *.apps.openshift4.example.com
That means all the URL's in a single LoadBalancer. I was able to access the console from below URL
https://console-openshift-console.apps.openshift4.example.com
Even another app was able to access from https://anotherapp.apps.openshift4.example.com
HA Proxy config file
frontend openshift-api-server
bind *:6443
default_backend openshift-api-server
mode tcp
option tcplog
backend openshift-api-server
balance source
mode tcp
server bootstrap 10.68.33.66:6443 check
server master1 10.68.33.63:6443 check
server master2 10.68.33.67:6443 check
server master3 10.68.33.68:6443 check
frontend machine-config-server68
bind *:22623
default_backend machine-config-server
mode tcp
option tcplog
backend machine-config-server
balance source
mode tcp
server bootstrap 10.68.33.66:22623 check
server master1 10.68.33.63:22623 check
server master2 10.68.33.67:22623 check
server master3 10.68.33.68:22623 check
frontend ingress-http
bind *:80
default_backend ingress-http
mode tcp
option tcplog
backend ingress-http
balance source
mode tcp
server worker1 10.68.33.64:80 check
server worker2 10.68.33.65:80 check
frontend ingress-https
bind *:443
default_backend ingress-https
mode tcp
option tcplog
backend ingress-https
balance source
mode tcp
server worker1 10.68.33.64:443 check
server worker2 10.68.33.65:443 check
But After reading the documentation https://docs.openshift.com/container-platform/4.2/installing/installing_vsphere/installing-vsphere.html#installation-network-user-infra_installing-vsphere I decided to use two load balancers. The API requires one load balancer and the default Ingress Controller needs the second load balancer to provide ingress to applications.
Now in this case I mapped the URL like below
10.68.33.62 api.openshift4.example.com
10.68.33.62 api-int.openshift4.example.com
And assuming IP of the second loadbalancer is 10.68.33.69
10.68.33.69 *.apps.openshift4.example.com
And HAProxy config for the first loadbalancer is only balancing the master nodes.
frontend openshift-api-server
bind *:6443
default_backend openshift-api-server
mode tcp
option tcplog
backend openshift-api-server
balance source
mode tcp
server bootstrap 10.68.33.66:6443 check
server master1 10.68.33.63:6443 check
server master2 10.68.33.67:6443 check
server master3 10.68.33.68:6443 check
frontend machine-config-server68
bind *:22623
default_backend machine-config-server
mode tcp
option tcplog
backend machine-config-server
balance source
mode tcp
server bootstrap 10.68.33.66:22623 check
server master1 10.68.33.63:22623 check
server master2 10.68.33.67:22623 check
server master3 10.68.33.68:22623 check
And the second load balancer is balancing only worker nodes because it will be serving only applications.
frontend ingress-http
bind *:80
default_backend ingress-http
mode tcp
option tcplog
backend ingress-http
balance source
mode tcp
server worker1 10.68.33.64:80 check
server worker2 10.68.33.65:80 check
frontend ingress-https
bind *:443
default_backend ingress-https
mode tcp
option tcplog
backend ingress-https
balance source
mode tcp
server worker1 10.68.33.64:443 check
server worker2 10.68.33.65:443 check
But unfortunately it's not working. Is my understanding correct? In a nutshell, I want to balance the Master Console and API's via first loadbalancer and the apps via second loadbalancer. How will I achieve it?
Thanks
Related
Is there any documentation how to use an external load balancer to load balance traffic on kubernetes API server.
Use case:
I don't prefer to use single master node ip/name in kubeconfig file and need a common name for all of the masters so that if one master is down , it sends traffic to other.
I have DNS name already pinting to loadbalancer ip , and loadbalancer is confifured with SSL certificate and kubernetes master backend nodes , but it results in an error:
"plan http request was sent to https server "
Somehow the load balancer is sending http request to kubernetes API server instead of https.
Turns out that it doenst work on L7 http , but works fine on L4 tcp.
The HAProxy configuration looks like:
frontend k8s-api
bind 192.168.0.150:443
bind 127.0.0.1:443
mode tcp
option tcplog
default_backend k8s-api
backend k8s-api
mode tcp
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server k8s-api-1 192.168.0.147:6443 check
server k8s-api-2 192.168.0.148:6443 check
server k8s-api-3 192.168.0.149:6443 check
I have haproxy doing ssl pass through communication with two httpd servers. I want to send the source ip of the client to the httpd servers.
I tried option forwardfor but it doesn't worked.
Haproxy version is 1.5.18 and the configuration is like this:
frontend localhttps
bind *:443
mode tcp
option tcplog
option forwardfor
default_backend httpsnodes
backend httpsnodes
mode tcp
option tcplog
balance source
cookie SERVERID insert indirect
option ssl-hello-chk
option forwardfor
server <host1> <ip1> cookie 01 check
server <host2> <ip2> cookie 02 check
I need some advise on how to setup haproxy. I have two web-servers up and running. For testing they run a simple node server on port 8080.
Now on my haproxy server I start haproxy which gives me the following:
$> /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg
[WARNING] 325/202628 (16) : Server node-backend/server-a is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 325/202631 (16) : Server node-backend/server-b is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 325/202631 (16) : backend 'node-backend' has no server available!
Just one note: If I do:
haproxy$> wget server-a:8080
I get the response from the node server.
Here is my haproxy.cfg:
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp
log global
option tcplog
option dontlognull
option http-server-close
# option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend www
bind *:80
default_backend node-backend
#---------------------------------------------------------------------
# round robin balancing between the various backends
#--------------------------------------------------------------------
backend node-backend
balance roundrobin
mode tcp
server server-a 172.19.0.2:8080 check
server server-b 172.19.0.3:8080 check
If I remove the check option it seems to work. Any suggestions how I can fix this checking mechanism of haproxy?
You need to get exact ip address of your server with the help of command
ifconfig
and correct the below address in your haproxy.cfg file:
172.19.0.2:8080
172.19.0.3:8080
or modify line like below
server server-a server-a:8080 check
server server-b server-b:8080 check
Remove "mode tcp" and change it to "mode http".
Im just guessing here but i suppose haproxy is doing a tcp check against your web server and the web server can not respond to it.
in "mode http" it checks the web server in http mode and expects a "response 200" for L4 check
and expects a string (whatever you defined) as a L7 check
eg. L4
backend node-backend
balance roundrobin
mode http #(NOT NEEDED IF DEFINED IN DEFAULTS)
option httpchk
server server-a 172.19.0.2:8080 check
server server-b 172.19.0.3:8080 check
eg. L7
backend node-backend
balance roundrobin
mode http #(NOT NEEDED IF DEFINED IN DEFAULTS)
option httpchk get /SOME_URI
http-check expect status 200
server server-a 172.19.0.2:8080 check
server server-b 172.19.0.3:8080 check
Another note related to #basickarl's comment on docker. If you are sending into a docker (docker-compose) instance (namely where you have multiple instances of service running) you likely need to define the docker resolver and use it for dns resolution on your backend:
resolver:
resolvers docker_resolver
nameserver dns 127.0.0.11:53
backend usage of resolver:
backend main
balance roundrobin
option http-keep-alive
server haproxyapp app:80 check inter 10s resolvers docker_resolver resolve-prefer ipv4
i tryied all this answers nothing works for me. only put the gateway IP of network work, for default bridge is 172.17.0.1.
In the servers put the : and with this haproxy connects with success.
My example of custom network with fixed ips and gateway:
----- haproxy config
backend be_pe_8545
mode http
balance roundrobin
server p1 172.20.0.254:18545 check inter 10s
server p2 172.20.0.254:28545 check inter 10s
----- docker app / network
docker_app: ...
networks:
public_network:
ipv4_address: 172.20.0.50
public_network:
name: public_network
driver: bridge
ipam:
driver: default
config:
- subnet: 172.20.0.0/24
gateway: 172.20.0.254
Is it possible to set up backend as Hostnames or URLs?
I want to use haproxy to balance both http and https requests to a pool of backends (TCP mode).
Since backends can change their IPs, I want to set backend servers as URLs.
...
defaults
mode tcp
option redispatch
...
frontend all
bind *:80
bind *:443
option tcplog
# DEFAULT
default_backend hostname_servers
backend hostname_severs
mode tcp
balance roundrobin
option ssl-hello-chk
server host1 host1.myapp.com check
server host2 host2.myapp.com check
...
server hostN hostN.myapp.com check
Thanks!
I do believe that functionality is now available in 1.6, http://blog.haproxy.com/2015/10/14/whats-new-in-haproxy-1-6/
From Link:
resolvers docker
nameserver dnsmasq 127.0.0.1:53
defaults
mode http
log global
option httplog
frontend f_myapp
bind :80
default_backend b_myapp
backend b_myapp
server s1 nginx1:80 check resolvers docker resolve-prefer ipv4
I have an haproxy setup which is NOT fronting an HTTP cluster. How can I ask haproxy which nodes are currently in the healthy list, and which are considered unhealthy?
I found the "stats" command, but that only works for HTTP traffic
You can add an additional port to your frontend (e.g. 8080) and enable stats for only that backend.
frontend
bind *:123,*:8080
acl is_stats dst_port 8080
use_backend stats if is_stats
default_backend your_TCP_backend
backend stats
stats enable