I have a container with IBM MQ (Docker image ibmcom/mq/9.2.2.0-r1) exposing two ports (9443 - admin, 1414 - application).
All required setup in OpenShift is done (Pod, Service, Routes).
There are two routes, one for each port.
https://route-admin.my.domain
https://route-app.my.domain
pointing to the ports accordingly (external ports are default http=80, https=443).
Admin console is accessible through the first route, hence, MQ is up and running.
I tried to connect as a client (JMS 2.0, com.ibm.mq.allclient:9.2.2.0) using standard approach:
var fctFactory = JmsFactoryFactory.getInstance(WMQConstants.WMQ_PROVIDER);
var conFactory = fctFactory.createConnectionFactory();
// ... other props
conFactory.setObjectProperty(WMQConstants.WMQ_HOST_NAME, "route-app.my.domain");
conFactory.setObjectProperty(WMQConstants.WMQ_PORT, 443);
and failed to connect. Also tried to redefine route as HTTP and use port 80, and again without success.
If it helps let's assume we use the latest version of MQ Explorer as a client.
Each time the same connection error appears:
...
Caused by: com.ibm.mq.MQException: JMSCMQ0001:
IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
...
Caused by: com.ibm.mq.jmqi.JmqiException:
CC=2;RC=2009;AMQ9204: Connection to host 'route-app.my.domain(443)' rejected.
[1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2009;AMQ9208:
Error on receive from host 'route-app.my.domain/10.227.248.2:443 (route-app.my.domain)'.
[1=-1,2=ffffffff,3=route-app.my.domain/10.227.248.2:443 (route-app.my.domain),4=TCP]],
3=route-app.my.domain(443),5=RemoteConnection.receiveTSH]
...
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9208:
Error on receive from host 'route-app.my.domain/10.227.248.2:443
Maybe, this article could give some hints about error code 2009, but still not sure what exactly affects connection errors from the OpenShift side.
Previously, I always connected to IBM MQ specifying a port value explicitly, but here is a bit different situation.
How to connect to IBM MQ in OpenShift cluster through TCP?
Configurations in OpenShift are as follows:
kind: Pod
apiVersion: v1
metadata:
name: ibm-mq
labels:
app: ibm-mq
spec:
containers:
- resources:
limits:
cpu: '1'
memory: 600Mi
requests:
cpu: '1'
memory: 600Mi
name: ibm-mq
ports:
- containerPort: 1414
protocol: TCP
- containerPort: 9443
protocol: TCP
containerStatuses:
image: 'nexus-ci/docker-lib/ibm_mq:latest'
---
kind: Service
apiVersion: v1
metadata:
name: ibm-mq
spec:
ports:
- name: admin
protocol: TCP
port: 9443
targetPort: 9443
- name: application
protocol: TCP
port: 1414
targetPort: 1414
selector:
app: ibm-mq
---
kind: Route
apiVersion: route.openshift.io/v1
metadata:
name: ibm-mq-admin
spec:
host: ibm-mq-admin.my-domain.com
to:
kind: Service
name: ibm-mq
weight: 100
port:
targetPort: admin
tls:
termination: passthrough
insecureEdgeTerminationPolicy: None
wildcardPolicy: None
---
kind: Route
apiVersion: route.openshift.io/v1
metadata:
name: ibm-mq-app
spec:
host: ibm-mq-app.my-domain.com
to:
kind: Service
name: ibm-mq
weight: 100
port:
targetPort: application
tls:
termination: passthrough
insecureEdgeTerminationPolicy: None
wildcardPolicy: None
---
UPDATE: Ended up with creating and deploying to OpenShift a small web-application receiving HTTP requests and interacting with MQ via JMS (put/get text messages), like:
POST /queue/{queueName}/send + <body>;
GET /queue/{queueName}/receive.
It interacts with MQ inside the OpenShift cluster using TCP, and accepts external HTTP connections as a regular web application.
Other solutions seem to take too much efforts, but I accepted one of them as it is theoretically correct and straightforward.
I'm not sure to fully understand your setup, but"Routes"only route HTTP traffic (On ports 80 or 443 onyl), not TCP traffic.
If you want to access your MQ server from outside the cluster, there are a few solutions, one is to create a service of type: "NodePort"
Doc: https://docs.openshift.com/container-platform/4.7/networking/configuring_ingress_cluster_traffic/configuring-ingress-cluster-traffic-nodeport.html
Your Service is not a NodePort Service. In your case, it should be something like
kind: Service
apiVersion: v1
metadata:
name: ibm-mq
spec:
type: NodePort
ports:
- port: 1414
targetPort: 1414
nodePort: 30001
selector:
app: ibm-mq
Then access from outside with anyname.<cluster domaine>:30001
And delete the useless corresponding route. As said before, I assumed you read in the doc I pointed to you that says that route only route HTTP traffic on port 80 or 443.
Doc: https://kubernetes.io/docs/concepts/services-networking/service/#nodeport
The following Java system property will be read by IBM MQ classes for JMS at 9.2.1 and higher to tell it to set the SNI header to the hostname of the remote system when initiating a TLS connection:
com.ibm.mq.cfg.SSL.OutboundSNI=HOSTNAME
To set this programmatically just use the System.setProperty method for example:
System.setProperty("com.ibm.mq.cfg.SSL.OutboundSNI","HOSTNAME");
NOTE: the string HOSTNAME is literal and not meant to be replaced by a actual hostname.
If you can not move to a com.ibm.mq.allclient.jar from 9.2.1 or later, then in 9.2.0.0 and later you could instead use com.ibm.mq.cfg.SSL.AllowOutboundSNI=NO, but this is deprecated in 9.2.1 and later.
Related
I'd like to launch a clustered Socket IO application in Kubernetes. When I create a service (whether NodePort or LoadBalancer) the client application keeps getting disconnected and it reconnects again with the following logs:
undefined
oah4g28zZCw36g1MAAAm
undefined
undefined
oac4g28zZCw36g1MFAAAx
undefined
and this happens rapidly.
However, when I connect to a single Pod directly, the problem goes away and the connection becomes stable.
How I am creating the service is by the following command:
kubectl expose deployment xxx --type=LoadBalancer --port=80 --target-port=3000
I know that something such as a KeepAlive or Timeout configuration is missing in the service, but how can I add those or better said properly configure the service for Socket IO?
You can use the sessionAffinity: ClientIP, which will manage the session from K8s service.
kind: Service
apiVersion: v1
metadata:
name: example
spec:
selector:
app: example
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 1800
just for ref : Does the ws websocket server library requires sticky session when it is used behind a load balancer?
I have the following service:
apiVersion: v1
kind: Service
metadata:
name: hedgehog
labels:
run: hedgehog
spec:
ports:
- port: 3000
protocol: TCP
name: restful
- port: 8982
protocol: TCP
name: websocket
selector:
run: hedgehog
externalIPs:
- 1.2.4.120
In which I have specified an externalIP.
I'm also seeing this IP under EXTERNAL-IP when running kubectl get services.
However, when I do curl http://1.2.4.120:3000 I get a timeout. However the app is supposed to give me a response because the jar running inside the container in the deployment does respond to localhost:3000 requests when run locally.
if you see the type of your service might be cluster IP try changing the type to LoadBalancer
apiVersion: v1
kind: Service
metadata:
name: http-service
spec:
clusterIP: 172.30.163.110
externalIPs:
- 192.168.132.253
externalTrafficPolicy: Cluster
ports:
- name: highport
nodePort: 31903
port: 30102
protocol: TCP
targetPort: 30102
selector:
app: web
sessionAffinity: None
type: LoadBalancer
something like this where type: LoadBalancer.
First of all you have to understand you cannot place any random address in your ExternalIP field. Those addresses are not managed by Kubernetes and are the responsibility of the cluster administrator or you. External IP addresses specified with externalIPs are different than the external IP address assigned to a service of type LoadBalancer by a cloud provider.
I checked the address that you mentioned in the question and it does not look like it belongs to you. That why I suspect that you placed a random one there.
The same address appears in this article about ExternalIP. As you can see here the address in this case are the IP addresses of the nodes that Kubernetes runs on.
This is potential issue in your case.
Another one is too verify if your application is listening on localhost or 0.0.0.0. If it's really localhost then this might be another potential problem for you. You can change where the server process is listening. You do this by listening on 0.0.0.0, which means “listen on all interfaces”.
Lastly please verify that your selector/ports of the services are correct and that you have at least one endpoint that backs your service.
I can't connect to an "external" (outside the mesh) Kafka service from inside the mesh. Inside the Istio mesh I have an Spring Boot app, which should connect to an platform Kafka service.
The Kafka service is reachable via an DNS name (I tried it, it is possible). To access the service I created an ServiceEntry with the following configuration:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: kafka
namespace: test
spec:
hosts:
- kafka-service.foo.baa
ports:
- number: 37000
name: tls-6
protocol: tls
- number: 36200
name: tls-5
protocol: tls
- number: 36201
name: tls-4
protocol: tls
- number: 36202
name: tls-3
protocol: tls
- number: 36203
name: tls-2
protocol: tls
- number: 36204
name: tls-1
protocol: tls
resolution: DNS
location: MESH_EXTERNAL
Sometime I get an connection, but sometime not and I don't know the issue and why it's working sometime.
These are some error message:
org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
Removing server kafka-service.foo.baa:37000 from bootstrap.servers as DNS resolution failed for kafka-service.foo.baa
I also tried to use this anntotation traffic.sidecar.istio.io/excludeOutboundPorts: 37000,36200,36201,36202,36203,36204,36205 to bypass the traffic, but this is not working as well.
The Bypass is only working, if I use the IP address instread of the DNS name as bootstrap server.
Please kindly help.
I am trying to setup K8S to work with two Windows Nodes (2019). Everything seems to be working well and the containers are working and accessible using k8s service. But, once I introduce configuration for readiness (or liveness) probes - all fails. The exact error is:
Readiness probe failed: Get http://10.244.1.28:80/test.txt: dial tcp 10.244.1.28:80: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
When I try the url from k8s master, it works well and I get 200. However I read that the kubelet is the one executing the probe and indeed when trying from the Windows Node - it cannot be reached (which seems weird because the container is running on that same node). Therefore I assume that the problem is related to some network configuration.
I have a HyperV with External network Virtual Switch configured. K8S is configured to use flannel overlay (vxlan) as instructed here: https://learn.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/network-topologies.
Any idea how to troubleshoot and fix this?
UPDATE: providing the yaml:
apiVersion: v1
kind: Service
metadata:
name: dummywebapplication
labels:
app: dummywebapplication
spec:
ports:
# the port that this service should serve on
- port: 80
targetPort: 80
selector:
app: dummywebapplication
type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: dummywebapplication
name: dummywebapplication
spec:
replicas: 2
template:
metadata:
labels:
app: dummywebapplication
name: dummywebapplication
spec:
containers:
- name: dummywebapplication
image: <my image>
readinessProbe:
httpGet:
path: /test.txt
port: 80
initialDelaySeconds: 15
periodSeconds: 30
timeoutSeconds: 60
nodeSelector:
beta.kubernetes.io/os: windows
And one more update. In this doc (https://kubernetes.io/docs/setup/windows/intro-windows-in-kubernetes/) it is written:
My Windows node cannot access NodePort service
Local NodePort access from the node itself fails. This is a known
limitation. NodePort access works from other nodes or external
clients.
I don't know if this is related or not as I could not connect to the container from a different node as stated above. I also tried a service of LoadBalancer type but it didn't provide a different result.
The network configuration assumption was correct. It seems that for 'overlay', by default, the kubelet on the node cannot reach the IP of the container. So it keeps returning timeouts and connection refused messages.
Possible workarounds:
Insert an 'exception' into the ExceptionList 'OutBoundNAT' of C:\k\cni\config on the nodes. This is somewhat tricky if you start the node with start.ps1 because it overwrites this file everytime. I had to tweak 'Update-CNIConfig' function in c:\k\helper.psm1 to re-insert the exception similar to the 'l2bridge' in that file.
Use 'l2bridge' configuration. Seems like 'overlay' is running in a more secured isolation, but l2bridge is not.
I want to create a loadBalancer service on kubernetes that exposes a large range of ports. As you can't do that on kubernetes yet (https://github.com/kubernetes/kubernetes/issues/23864).
I have manually entered a range of port by having a yaml file in the following format:
apiVersion: v1
kind: Service
metadata:
name: service
spec:
ports:
- name: port10000
port: 10000
protocol: UDP
.
.
.
- name: port40000
port: 40000
protocol: UDP
selector:
app: app-label
type: LoadBalancer
I get the following error:
Error from server (InternalError): error when creating "service-udp.yml": Internal error occurred: failed to allocate a nodePort: range is full
Is it possible to increase the range of ports available for a service? And if so, how?
This is controlled by the --service-node-port-range portRange argument to kube-apiserver - the way to change that depends on your environment.
Keep in mind that nodePorts are meant to be used by load balancers as building blocks. So what you are trying to do is most likely not the best practice.
Hope this helps..