We have a Websphere app server with one cluster containing 2 servers. Each server contains our webapp. Each webapp/server talks to an single NCipher server though sockets. The NCipher server is listening for communications from each webapp/server on a common port.
If we reboot each of the Websphere servers in the cluster individually then we find that one connects straightaway but the other takes a variable amount of reboots before it connects. What I find confusing is why one connects immediately but the other doesn't.
What I have noticed is that connections are not being closed in the webapp. Not sure what impact this may have given that one of the servers is connecting fine once rebooted and the other doesn't.
Even though not closed I guess the connections do timeout, although closing them is definitely the correct thing to do.
Would anybody have any idea as to why this may be occuring ?
Related
I have an Ubuntu VM installed on a client's VMware system. Recently, the client's IT informed us that his firewall has been detecting consistent potential port scans to our VM's internal IP address (coming from 87.238.57.227). He asked if this was part of a known package update process on our VM.
He sent us a firewall output where we can see several instances of the port scan, but there are also instances of our Ubuntu VM trying to communicate back to the external server on port 37258 (this is dropped by the firewall).
Based on a google lookup, the hostname of the external IP address is "feris.postgresql.org", with the ASN pointing to a European company called Redpill-Linpro. As far as I can tell, they offer IT consulting services, specializing in open source software (like PostgreSQL, which is installed on our VM). I have never heard of them before though and have no idea why our VM would be communicating with them or vice-versa. I'm also not sure if I'm interpreting the IP lookup information correctly: https://ipinfo.io/87.238.57.227
I'm looking for a way to confirm or disprove that this is just our VM pinging for a standard postgres update. If that's the case I'd like to restrict this behaviour. We would prefer to do these types of updates manually and limit the communication outside of the VM to what is strictly necessary for the functionality of our application.
Update
I sent an email to Redpill's abuse account. They responded quickly saying that the server should not be port scanning anyone and if it appears that way, something is wrong.
The server is part of a cluster of machines that serves apt.postgresql.org among other postgres download sites. I don't think we have anything like ansible or puppet installed that would automatically check for updates but I will look into that to make sure. I'm wondering if Ubuntu reaching out to update the MOTD with the number of available packages would explain why our VM is trying to reach out to the external postgres server?
The abuse rep said in any case there should only be outgoing connections from the VM, not incoming. He asked for some additional info so I will keep communicating with him and try to update this post accordingly
My communication with the client's IT dropped off so I did not get a definitive answer on this, but I'll provide some new details:
I reached out to the abuse email for Redpill-Linpro. He got back to me and confirmed the server corresponding to the detected IP address is part of a cluster that hosts postgres download sites, including apt.postgresql.org. He was surprised to learn we had detected a port scan from their server and seems eager to figure out why that is happening.
He asked if the client IT could pass along some necessary info for them to set up tracking on that server. But the client IT never got back to me. I think he was satisfied that it wasn't malicious and stopped pursuing it.
Here's one of the messages the abuse rep sent me that may be relevant:
That does look a lot like the tcp to the apt download server yes. It's
strange that your firewall reports that many incoming connections, but
they could be fallout from some connection tracking that's not
operating as intended. The timing appears to be matching up more or
less perfectly. And there should definitely not be any ping-back
connections from it.
Since you appear to be using the http version of the server (and not https) bringing the data in cleartext, they should be able to just
dump the TCP connection contents and verify exactly what it does. But
I bet they are going to see a number of http requests initiated by the
apt client that is checking for updates.
We made a chat module in our project using socket.io. When the load is balanced and the manual deployed, if socket connections are switched to different servers, socket connections are disconnected and the messaging events are partially not processed. I solved the load balance problem with socket.io-redis library. It acts as a gateway and solves this problem thanks to redis.
Another problem is that when I deploy it manually, the pid of the servers changes and socketio connections are instantly disconnected on the client and then it is not connected even though it says connected.
Do you think that using tools such as Travis CI solves the problems in manual deploy process?
Another question is, if a system that goes to 3 servers with load balance then goes back to 2 servers, the socket connections will be closed again, what method may be required to solve this? I thought of separating the socket.io service from the monolithic structure and keeping it on a single server, and scaling the server vertically when the load increased.
We are using an Aws Elastic Beanstalk(EBS), it automatically performs load balance.
I have an application with a server written in F# and serve web files using suave. I remote login using powershell into another machine in the network to run the application (The application is also in one of the network drives). I do that because that machine have access to third party APIs needed for the server. Now when I do [IPAddress_Of_Remote_Machine]/[html_file] or [name_of_pc]/[html_file] then chrome is waiting forever and doesn't ever return the webpage. This wasn't happening before and I ran into this problem recently. I opened a different port and used it instead of the default one 80. This made things work but the problem keeps showing up after a couple of days. I don't think it's a firewall issue but I'm clueless to why this is happening.
When running netstat -an, this is what I get (I hid the IP address):
As you can see all of the connections are either in CLOSE_WAIT or ESTABLISHED but not LISTENING. All of these TCP connections is probably because I have PhantomJS and two other APIs running in the application as well. However the loop back address is also open on the same port 5959:
I'm not sure what is difference between these two but when using PortQryUI to query the remote server it returns a success!
I have already made an inbound rule for port 5959 on the server so it should be allowed. The web page is stuck at Waiting for [name_of_pc]. Also, sometimes this problem disappears and everything works fine.
What is the potential problem behind this? Why would this happen all of a sudden?
UPDATE:
I re-ran the application today and it's working correctly. It could be that something is dynamically set within the firewall? Not really sure what is going on. The machine I'm running the server on has a bunch of applications running on it as well so maybe there is an external process that is affecting it?
I made a hello world app with Suave and deployed it on the network drive to test if it's going to work. I opened inbound rule for port 6001
Then I ran the app:
However, it's still not working and this time it says the site cannot be reached when I do: http://[name_of_pc]:6001.
Moving this to an answer so that it can be closed:
Could you post the bindings section of your suave cfg? I'm guessing you know where that is since you are using a non-standard port but if you need don't, search for HttpBinding. I suspect you will find it pointing to 127.0.0.1 which is not good enough for remote access. You could try changing it to 0.0.0.0 or to the server's actual IP address. I would try 0.0.0.0 first for the flexibility it provides
I have a simple node.js client and server programs running on one machine and when I try to connect to the server with second instance of client program simultaneously I get EADDRINUSE, Address already in use error. Is it possible to have two or more TCP based socket client connections (created with createConnection) to one server (created with createServer) on the same machine, or only one client program can be connected to the server at the same time?
Yes, it's possible. Infact, very common. Many applications open dozens, or hundreds of connections to the same server. It sounds like your client program is binding on a port. Only the server should be binding on a port. You should verify.
The client will usually use a random port between 1024-65535, as assigned by your OS. You don't need to worry about it. Since the client is initiating a connection to the server, the server port must unique to one program. Which is why your problem signifies you are trying to start the server twice. Please see http://www.tcpipguide.com/free/t_TCPIPClientEphemeralPortsandClientServerApplicatio.htm
Recently we got a new server at the office purely for testing purposes. It is set up so that we can access it from any computer.
However today our ip got blocked from one of our other sites saying that our ip has been suspected of having a virus that sends spam emails. we learned this from the cbl http://cbl.abuseat.org/
So of course we turned the server off to stop this. The problem is the server must be on to continue developing our application and to access the database that is installed on it. Our normal admin is on vacation and is unreachable, and the rest of us are idiots(me included) in this area.
We believe that the best solution is to remove it from connecting to the internet but still access it on the lan. If that is a valid solution how would this be done or is there a better way? say blocking specified ports or whatever.
I assume that this server is behind a router? You should be able to block WAN connections to the server on the router and still leave it open to accepting LAN connection. Or you could restrict the IPs that can connect to the server to the development machines on the network.