I have a docker image listed in my gitlab ci services list. When I make an htpp request using curl to my docker service url everything works fine. But when I run my tests which makes an http request using axios to the service docker image url it says connection refused here is the exact message connect EINVAL 0.0.31.129:80 - Local (0.0.0.0:0)
The thing is that I was using a service on my registry running on specific port and the runner was trying to connect to the registry at port 80. Because since I did specify the port the runner don't exactly know what protocol to use. So, the runner pick default port for http which is 80. So, the fix will be to add http protocol to the registry url.
Related
SUMMARY
I have installed zabbix on OpenShift cluster. I am trying to monitor a host(vm) outside the cluster but the zabbix server is unable to connect to it. In the /etc/zabbix/zabbix_agentd.conf file I have mentioned the DNS name of the server zabbix-server but it looks like there server is trying to connect through a different public IP. I am not sure what this IP is.
OS / ENVIRONMENT / Used docker-compose files
I applied the kubernetes.yaml file present in this repo - https://github.com/zabbix/zabbix-docker/blob/6.2/kubernetes.yaml - on an OpenShift cluster.
CONFIGURATION
In the /etc/zabbix/zabbix_agentd.conf file Server=zabbix-server.
STEPS TO REPRODUCE
Apply the kubernetes.yaml file on Openshift cluster and try to monitor any external vm.
EXPECTED RESULTS
The zabbix server should be able to connect to the vm.
ACTUAL RESULTS
Zabbix server logs.
Defaulted container "zabbix-server" out of: zabbix-server, zabbix-snmptraps
\*\* Updating '/etc/zabbix/zabbix_server.conf' parameter "DBHost": 'mysql-server'...added
287:20230120:060843.131 Zabbix agent item "system.cpu.load\[all,avg5\]" on host "Host-C" failed: first network error, wait for 15 seconds
289:20230120:060858.592 Zabbix agent item "system.cpu.num" on host "Host-C" failed: another network error, wait for 15 seconds
289:20230120:060913.843 Zabbix agent item "system.sw.arch" on host "Host-C" failed: another network error, wait for 15 seconds
289:20230120:060929.095 temporarily disabling Zabbix agent checks on host "Host-C": interface unavailable
Logs from the agent installed on the vm.
350446:20230122:103232.230 failed to accept an incoming connection: connection from "9.x.x.219" rejected, allowed hosts: "zabbix-server"
350444:20230122:103332.525 failed to accept an incoming connection: connection from "9.x.x.219" rejected, allowed hosts: "zabbix-server"
350445:20230122:103432.819 failed to accept an incoming connection: connection from "9.x.x.210" rejected, allowed hosts: "zabbix-server"
350446:20230122:103533.114 failed to accept an incoming connection: connection from "9.x.x.217" rejected, allowed hosts: "zabbix-server"
If I add this IP in /etc/zabbix/zabbix_agentd.conf it will work. But what IP is this? Is this a service? Or any node/pod IP? It keeps on changing. Everytime I cannot change this id in the conf file. I need something more stable.
Kindly help me out with this issue.
So I don't know zabbix. So I have to make some educated guesses both in how the agent works and how the server works.
But, to summarize, unlike something like docker compose where you are running the zabbix server on a known server, in Openshift/Kubernetes you are deploying into a cluster of machines with their own networking. In other words, the whole point of OpenShift is that OpenShift will control where the application's pod gets deployed and will relocate/restart that pod as needed. With a different IP every time. (And the DNS name is meaningless since the two systems aren't sharing DNS anyway.) Most likely the IP's you are seeing are the pod's randomly assigned IPs.
So, what are you to do when you have a situation like yours where an external application requires a predicable IP? Well, option 1, is to remove that requirement. Using something like a certificate is obviously more secure and more reliable than depending on an IP anyway. But another option is to use an egress IP. This is a feature of OpenShift where you essentially use a proxy to provide an external application with a consistent IP.
I'm trying to setup a remote EJB call between 2 WebSphere Liberty servers deployed in k8s.
Yes, I'm aware that EJB is not something one would want to use when deploying in k8s, but I have to deal with it for now.
The problem I have is how to expose remote ORB IP:port in k8s. From what I understand, it's only possible to get it to work if both client and remote "listen" on the same IP. I'm not a network expert, and I'm quite fresh in k8s, so maybe I'm missing something here, that's why I need help.
The only way I got it to work is when I explicitly set host on remote server to it's own IP address and then accessed it from client on that same IP. This test was done on Docker host with macvlan0 network (each container had it's own IP address).
This is ORB setup for remote server.xml configuration:
<iiopEndpoint id="defaultIiopEndpoint" host="172.30.106.227" iiopPort="2809" />
<orb id="defaultOrb" iiopEndpointRef="defaultIiopEndpoint">
<serverPolicy.csiv2>
<layers>
<!-- don't care about security at this point -->
<authenticationLayer establishTrustInClient="Never"/>
<transportLayer sslEnabled="false"/>
</layers>
</serverPolicy.csiv2>
</orb>
And client server.xml configuration:
<orb id="defaultOrb">
<clientPolicy.csiv2>
<layers>
<!-- really, I don't care about security -->
<authenticationLayer establishTrustInClient="Never"/>
<transportLayer sslEnabled="false"/>
</layers>
</clientPolicy.csiv2>
</orb>
From client, this is JNDI name I try to access it:
corbaname::172.30.106.227:2809#ejb/global/some-app/ejb/BeanName!org\.example\.com\.BeanRemote
And this works.
Since one doesn't want to set fixed IP when exposing ORB port, I have to find a way to expose it dynamically, based on host IP.
Exposing on 0.0.0.0 does not work. Same goes for localhost. In both cases, client refuses to connect with this kind of error:
Error connecting to host=0.0.0.0, port=2809: Connection refused (Connection refused)
In k8s, I've exposed port 2809 through LoadBalancer service for remote pods, and try to access remote server from client pod, where I've set remote's service IP address in corbaname definition.
This, of course, does not work. I can access remote ip:port by telnet, so it's not a network issue.
I've tried all combinations of setup on remote server. Exporting on host="0.0.0.0" results with same exception as above (Connection refused).
I'm not sure exporting on internal IP address would work either, but even if it would, I don't know the internal IP before pod is deployed in k8s. Or is there a way to know? There is no env. variable with it, I've checked.
Exposing on service IP address (with host="${REMOTE_APP_SERVICE_HOST}") fails with this error:
The server socket could not be opened on 2,809. The exception message is Cannot assign requested address (Bind failed).
Again, I know replacing EJB with Rest is the way to go, but it's not an option for now (don't ask why).
Help, please!
EDIT:
I've managed to get some progress. Actually, I believe I've successfully called remote EJB.
What I did was add hostAliases in pod definition, which added alias for my host, something like this:
hostAliases:
- ip: 0.0.0.0
hostnames:
- my.host.name
Then I added this host name to remote server.xml:
<iiopEndpoint id="defaultIiopEndpoint" host="my.host.name" iiopPort="2809" />
I've also added host alias to my client pod:
hostAliases:
- ip: {remote.server.service.ip.here}
hostnames:
- my.host.name
Finally, I've changed JNDI name to:
corbaname::my.host.name:2809#ejb/global/some-app/ejb/BeanName!org\.example\.com\.BeanRemote
With this setup, remote server was successfully called!
However, now I have another problem which I didn't have while testing on Docker host. Lookup is done, but what I get is not what I expect.
Lookup code is pretty much what you'd expect:
Object obj = new InitialContext().lookup(jndi);
BeanRemote remote = (BeanRemote) PortableRemoteObject.narrow(obj, BeanRemote.class);
Unfortunatelly, this narrow call fails with ClassCastException:
Caused by: java.lang.ClassCastException: org.example.com.BeanRemote
at com.ibm.ws.transport.iiop.internal.WSPortableRemoteObjectImpl.narrow(WSPortableRemoteObjectImpl.java:50)
at [internal classes]
at javax.rmi.PortableRemoteObject.narrow(PortableRemoteObject.java:62)
Object I do receive is org.omg.stub.java.rmi._Remote_Stub. Any ideas?
Solved it!
So, the first problem was resolving host mapping, which was resolved as mentioned in edit above, by adding host aliases id pod definitions:
Remote pod:
hostAliases:
- ip: 0.0.0.0
hostnames:
- my.host.name
Client pod:
hostAliases:
- ip: {remote.server.service.ip.here}
hostnames:
- my.host.name
Remote server then has to use that host name in iiop host definition:
<iiopEndpoint id="defaultIiopEndpoint" host="my.host.name" iiopPort="2809" />
Also, client has to reference that host name through JNDI lookup:
corbaname::my.host.name:2809#ejb/global/some-app/ejb/BeanName!org\.example\.com\.BeanRemote
This setup resolves remote EJB call.
The other problem with ClassCastException was really unusual. I managed to reproduce the error on Docker host and then changed one thing at a time until the problem was resolved. It turns out that the problem was with ldapRegistry-3.0 feature (!?). Adding this feature to client's feature list resolved my problem:
<feature>ldapRegistry-3.0</feature>
With this feature added, remote EJB was successfully called.
I'm trying to set up a testing CI with Github Actions and Docker Compose. You can see the repository here.
I have a frontend on port 3000 which communicates with a backend on port 4000. I am using a testing library (Cypress) that launches predetermined tasks in an emulated browser against the frontend.
My docker setup works locally, however I can't seem to get the networking / port mapping working correctly in the Github runner. The frontend service can't be found on http://localhost:3000.
NOTE: I am using network_mode: host to simplify the environment.
How can I configure the Github workflow to successfully connect to the frontend application on the host network on port 3000?
Hopefully I'm doing something wrong, I've read all documentation and scoured forums but can't seem to get to the bottom of an issue I'm experiencing. I'm using OSX btw.
Things that are working:
Connect to cloud SQL from local OS using proxy via either TCP or Socket
Connect to cloud SQL from local OS using proxy in container via TCP
Connect to cloud SQL from GKE using proxy in the same pod via TCP
Things that are not working:
Connect to cloud SQL from local OS using proxy in contain via sockets
Connect to cloud SQL from GKE using proxy in the same pod via socket
I suspect both of these problems are actually the same problem. I'm using this command to run the proxy inside of the container:
docker run -v [PATH]:/cloudsql \
gcr.io/cloudsql-docker/gce-proxy /cloud_sql_proxy -dir=/cloudsql \
-instances=[INSTANCE_CONNECTION_NAME] -credential_file=/cloudsql/[FILE].json
And the associated socket is being generated with the directory. However when I attempt to connect I get the following error:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/cloudsql/node-sql:us-central1:nodedb' (61)
The proxy doesn't generate a new line when I try to connect which makes me think that it's not receiving the request, it simply says Ready for new connections and waits.
Any idea what's going wrong, or how I could troubleshoot this further?
For "Connect to cloud SQL from GKE using proxy in the same pod via socket" can you please follow the tutorial at https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine? We have a working WordPress example there that has the cloudsql-proxy as a sidecar container (i.e. in the same Pod, but over TCP).
I don't think you can do "in the same pod via socket" unless you’re running multiple processes in a single container (which you shouldn’t as a best practice). If you do a sidecar container, you can use TCP, so you don’t need a unix socket (moreover, I'm not sure how you’d share files between containers of a Pod).
Also, the docker run -v /local.sock:/remote.sock (I think) will be creating a file/directory locally as /local.sock and making that available inside the container as /remote.sock. This might not work because the docker-engine doesn't know that /local.sock is meant to be a Unix socket and it creates a regular file.
I'm having trouble completing a web deploy from VS2013 as I posted in VS to Azure Publish failure: Socket Error 10054
Apparently web deploy uses port 8172, which I cannot telnet to at the moment despite rules being present in my windows firewall allowing the port through.
I am using the commands telnet localhost 8172 and telnet 192.168.0.1 8172 and it says it cannot connect to the host on port 8172.
I'm not sure this is the issue because I can complete a web deploy to Azure in my other projects? But in any event I'm confused as to why I cannot telnet to the port.
Is my syntax wrong?
JK
If you're using Azure you need to open up a port in the 'Network group' settings.
If you're using CloudFlare or some other kind of proxy, make sure to connect with the IP address of the VM or the MS supplied domain name and not your 'website domain name'. CloudFlare won't proxy through any old random port.