I'm building a message broker which communicates with clients over ZeroMQ PUSH/PULL sockets and has the ability to exclude clients from messages they're not subscribed to from the server side (unlike ZeroMQ pub/sub which excludes messages on the client side).
Currently, I implement it in the following way:
Server: Binds ZeroMQ PULL socket on a fixed port
Client: Binds a ZeroMQ PULL socket on a random or fixed port
Client: Connects to the server's PULL socket and sends a handshake message containing the new client's address and port.
Server: Recieves handshake from client and connects a PUSH socket to the client's PULL server. Sends handshake response to the client's socket.
Client: Recieves handshake. Connected!
Now the client and server can communicate bidirectionally and the server can send messages to only a certain subset of clients. It works great!
However, this model doesn't work if the clients binding PULL sockets are unable to open a port in their firewall so the server can connect to them. How can I resolve this with minimal re-architecting (as the current model works very well when the firewall can be configured correctly)
I've considered the following:
Router/dealer pattern? I'm fairly ignorant on this and documentation I found was sparse.
Some sort of transport bridging? The linked example provides an example for PUB/SUB.
I was hoping to get some advice from someone who knows more about ZeroMQ than me.
tl;dr: I implemented a message broker that communicates with clients via bidirectional push/pull sockets. Each client binds a PULL socket and the server keeps a map of PUSH sockets so that it can address specific subscribers. How do I deal with a firewall blocking the client ports?
You can use the router/dealer to do this like you say. By default the ROUTER socket tracks every connection it has. The way it does this is by having the caller stick the connection identity information in front of each message it recieves. This makes things like pub/sub fairly trivial as all you need to do is handle a few messages server side that the DEALER socket sends it. In the past I have done something like
1.) Server side is a ROUTER socket. The ROUTER handles 2 messages from DEALER sockets SUB/UNSUB. This alongside the identity info sent as the first part of a frame allows the router to know the messages that a client is interested in.
2.) The server checks the mapping to see which clients should be sent a particular type of data using the map and then forwards the message to the correct client by appending the identity again to the start of the message.
This is nice in that it allows a single port to be exposed on the server. Client side we do not need to expose ports, simply just connect to the server ROUTER socket.
See https://zguide.zeromq.org/docs/chapter3/ for more info.
Related
I am working on a product which uses ZeroMQ (version 4.0.1).
The server and client communicate based on ZeroMQ ROUTER-socket.
To read socket events, server and client also create socket-monitor sockets (PAIR). There are three ports on which server binds and listens. Out of these three ports, one port is in a non-secured mode. Other two ports are using md5-authentication.
The issue I am facing is that, both the server and the client spontaneously receive socket disconnect for one of the secure port sockets (please see a log below). I have checked multiple times that server and client both have L3 reachability to each other.
What else I should check for?
What really triggers this error scenario?
zmq_print_callback:ZmQ: int zmq::stream_engine_t::read(void*, size_t):923
Stream engine recv():
TCP socket (187) to unknown:0 was disconnected
with error 107 [Transport endpoint is not connected]
Below sequence of events can trigger this error on server
Server receives ACCEPTED event for clientY and gets FD1.
Link-flap/network issue happens and clientY disconnects but server does not receive this disconnect.
Network recovers and clientY connects back to server.
Server receives ACCEPTED event for clientY and gets FD2. However, packets sent to this sockets does not go out of the server.
After 1 min or so, clientY receives "Transport endpoint is not connected error" for FD1.
Application can use this to treat as client disconnect.
Here is my use case:
I have two endpoints: one with MQ and the second with TCP/IP
I have to replace a legacy server which accepts queries from remote TCP/IP clients. Once the socket is open with the client, data is exchanged in both sides. the server sends asynchronously MQ data through TCP/IP and receive data from clients asynchronously also. Each data message sent has to be acknowledged. The constraint here is that I have to use the same socket.
I created two routes
from("netty4:tcp://ipAddress:port?sync=true").to("wmq:queue:toQueue")
from("wmq:queue:fromQueue").to("netty4:tcp://ipAddress:port?sync=true")
I start the first queue to receive session open request from clients and then I start the second route to start sending data but I cannot use the same channel.
I tried to get the remote port of the first route and used it in the second route but I have a ConnectException because netty4 tries to open a new socket which is already open.
I found that netty4 can be used asynchronously using the AsyncProcessor but I didn't find any example dealing with my use case.
The only idea I found is that I have to create a standalone server which open the sockets with the clients and make it communicate with the two endpoints.
Is there any way to implement this situation using camel only?
any help on this subject is really appreciated.
Your code won't be able to run as it is for your use case. I also suspect you are trying to use Camel as IP server framework and not an integration in this case.
Lets review Apache Camel's concept of producers and consumers. In the integration world we talk about client and servers as consumers and producers. This might seem like a language difference until you realise a consumer(typically a client) can also be a producer(server).
Some helpful definitions:
1. Producer: A producer is an entity capable of creating and sending a message to an endpoint. A typical example would be code like .to("file:data/outbox") as this produces a file.
2. Consumer: A consumer is an entity that receives messages produced by a producer, it wraps these messages in an exchange and sends them to be processed. A typical example would be code like from(jms:topic:xmlOrders)
A rule of thumb is that typically consumers are the source of the messages being routed.
BIG NOTE:
These two definitions are not set in stone a producer can also be an endpoint using the from and a consumer can be an endpoint using the to.
So in your case let's break up the route:
from("netty4:tcp://ipAddress:port?sync=true").to("wmq:queue:toQueue")
In this route you are creating a Netty server that sends a message to a queue. Here your netty endpoint acts as a consumer(yes it is in the from clause) however this creates a Netty4 Server at the IP address and endpoint you specified. This then send a message to another consumer which is the MQ client which act as a consumer again. So two consumers? Where is the producer? The client connecting to the netty server will act as producer.
Let's look at the second piece of the route:
from("wmq:queue:fromQueue").to("netty4:tcp://ipAddress:port?sync=true")
Here you are creating a client/consumer for the MQ services and then creating a client/producer to the netty server. Essentially you are creating a NEW client here that connects to the SERVER you created in the first route.
So in short your route creates a Netty server that send a message to MQ then creates a MQ client that sends a message to a Netty client which connects to the server you have created. It wont work like this.
Go read about message exchange patterns for further reading, but I would suggest that if you are just using Netty and MQ then maybe Camel is a bit overkill as it is a integration platform and not a IP server platform.
I am trying to write a service discovery protocol for my communication framework. I am planning to do the following steps:
Server Side
Creating a multicast socket
Join a group
Broadcast(Send multicast message with some heart-beat [UDP Multicast Protocol]. The broadcast data contains server IP and some port which will be used to process client registration requests
Open thread to receive TCP packets from Client on that port to accept registrations
Client Side
Creating Multicast socket
Join the same multicast group as server to receive messages
Check for relevant broadcast from service
Get data from the broadcast packet (Data is server IP and Port)
Make a TCP Call to server(IP and port received) in the form of registration request.
Regarding security vulnerability, I can provide certificates in client registration step to protect that, but my main concern is in service broadcast.
If I broadcast server IP and port in a heartbeat that is very dangerous to attacks like port flooding etc.(I can provide some security measure but heavy-weight encryption and decryption will make service discovery lag). Is there any alternative design?
If I broadcast server IP and port in a heartbeat that is very dangerous to attacks like port flooding etc. (I can provide some security measure but heavy-weight encryption and decryption will make service discovery lag). Is there any alternative design?
An alternative design would be to send a broadcast from a client to discover the server. Just like DHCP and other services work: client broadcasts a "discovery" message, so every sever might response with a service "offer". Please find below more info:
https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol
Regarding the "discovery lag" you mention. Having client send the first "discovery" packet should reduce the lag to a minimum, so that is another plus to "reverse" your model.
When deploying a web application running on a traditional web server, you usually restart the web server after the code updates. Due to the nature of HTTP, this is not a problem for the users. On the next request they will get the latest updates.
But what about a WebSocket server? If I restart or kill the old process all connected users will get disconnected. So my question is, what kind of strategy have you used to deploy a WebSocket server smoothly?
You're right, every connected user will be disconnected if the server restarts.
I think the less bad solution is to tell to the client to reconnect in the onClose method of the client.
WebSockets is just a transport mechanism. Libraries like socket.io exist to build on that transport -- and provide heartbeats, browser fallbacks, graceful reconnects and handle other edge-cases found in real-time applications.
In our WebSocket-enabled application, socket.io is central to ensuring our continuous deployment setup doesn't break users' active socket connections.
If clients are connected directly to sever that does all sockets networking and application logic, then yes - they will be disconnected, due to TCP layer that holds connection.
If you have gateway that clients will be connecting to, and that gateway application is running on another server, but will communicate and forward messages to logical server, then logical server will send them back and gateway will send back to client responses. With such infrastructure, you have to implement stacking of packets on gateway until it will re-establish connection with logical server. Logical server might notify gateway server before restart. That way client will have connection, it will just wont receive any responses.
Or you can implement on client side reconnection.
With HTTP, every time you navigate away, browser actually is creating socket connection to server, transmits all data and closes it (in most cases). And then all website data is local, until you navigate away.
With WebSockets it is continuous connection, and there is no reconnection on requests. Thats why you have to implement simple mechanics when WebSockets getting closing event, you will try to reconnect periodically on client side.
It is more based on your specific needs.
I have 2 sip clients on the same computer.
Both of them is registering to a server that is running on port 5060.
For the first client the UDP is on port 5060 and for the other is 5061. When I come from one client to another, after the ringing part i receive the error:
only one usage of each socket address is normally permited.
Got any ideas why I got this error?
Your server and client are both trying to use port 5060, hence the error message. Change the first client to use 5062 or something else.
Also, 5061 is normally used for secured SIP (normal listening port + 1 in the proxy/server). Do not use it for the second client.
It means you're clients are both trying to claim the same socket for the communication channel, or the server is trying to reclaim the socket given to client A, to reuse it for client B.
The software handeling the socket, should be smart enough to rely on the OS to assign port numbers instead of hardcoding the port numbers in the code, this is a 100% guarantee for socket issues.