It is not clear to me why some papers use one or another term. I think they are the same thing: maybe atomic multicast is actually atomic broadcast that is implemented using IP multicast (like ring Paxos).
The term Atomic Broadcast is more related to a single central entity, usually called a sequencer, which is enforcing and sending a total ordered set of messages, usually called the event-stream. How it sends the messages (broadcast, multicast, unicast, tcp, etc.) is not its main characteristic, or at least it shouldn't be. Adding to what #jop has said, there are big technical differences between UDP broadcast and UDP multicast when it comes to the TCP/IP protocol. For example:
Multicast can travel across subnets, broadcast cannot
Multicast usually requires IGMP, broadcast does not
Most kernel-bypass network cards will accelerate multicast, but not broadcast
That does not mean that UDP broadcast should never be used. Some networks might not support multicast but will support broadcast. Ideally a messaging system will support several transport mechanisms:
UDP Multicast
UDP Unicast
UDP Broadcast
TCP
Memory
Shared Memory
DUAL (TCP + UDP)
For an example of an atomic broadcast messaging system which is not tied to any specific transport mechanism, you can check CoralSequencer.
Disclaimer: I'm one of the developer of CoralSequencer.
In distributed systems theory, it is not related to using IP multicast or any other implementation detail. Actually, most of the time it is a matter of personal preference of the author and you can safely assume that they mean the same.
In detail, to be strict, when you say multicast you are assuming that not all processes are necessarily intended to receive all messages. When you say broadcast, you are assuming that all processes are targeted by all messages. The ambiguity arises as follows: As often multicast algorithms are layered on top of membership management, an abstract protocol that multicasts to all members in the context of a view is pretty much indistinguishable from one that broadcasts to all processes in the system model. Hence, you can describe it as either multicast or broadcast. It really depends on the context.
Related
On my machine, I have 2 interfaces connected to another machine with 2 interfaces as well. I want to use both interfaces at the same time to transfer data. From SCTP view, each machine is an endpoint. So, I used a one-to-one socket. On the server side, I tried to bind INADDR_ANY as well as bind() the first and bindx() the second. On the client side, I tried connect() and connectx(). Whatever I tried, SCTP use only one of the two interfaces at a given time.
I also tested the sctp function on Iperf and the test app in the source code. Nothing works.
What am I missing here? Do you have to send each packet by hand from one or the other address and to one or the other address?
There surely must have a function where you can build several streams where each stream allows the communication between a pair of specific addresses. Then when you send a packet, SCTP chooses automatically which stream to send the packet in.
Thanks in advance!
What you are asking for called concurrent multipath transfer, feature that isn't supported by SCTP (at least not per RFC 4960).
As described in RFC 4960 by default SCTP transmits data over the primary path. Other paths are meant to be monitored by heartbeats and used when transmission over primary path fails.
The routing protocol OSPF has a configuration command as "network " to help determine which network interface to operate on. And this command can be used to multiple times on different networks. My question is if this means the OSPF socket, or, raw socket, can bind to multiple addresses? Per my understanding, each socket has only one protocol control block(PCB), and each PCB has only one address. I'm only looking for high level explanation. For example, if OSPF socket can have multiple PCBs, or, OSPF's PCB can have multiple addresses.
That depends on the implementation.
You don't need customized sockets to use different interfaces and you don't need to work only with one socket. The implementation can use N raw sockets each one bound to one of N interfaces. Then the implementation can have one or N threads handling those N sockets.
Also it could use packet sockets but it doesn't make much sense.
Additionally it could use libpcap for receiving messages and sending messages.
Look at Quagga: https://en.wikipedia.org/wiki/Quagga_(software)
If I want to use (UDP) sockets as an inter-process communication mechanism on a single PC, are there restrictions on what I can set up due to the two endpoints having the same IP address?
I imagine that in order to have two processes A and B both listening on the same IP/port address, SO_REUSADDR would be necessary - correct? And even though that might conceptually allow for full duplex comms over a single socket, there are other questions I have if I try to go full duplex:
would I end up receiving my own transmissions, and have to filter them out?
would I be exposing myself to other processes injecting spurious or malicious data into my sockets due to the use of SO_REUSEADDR... or do I face this possibility simply by using (connectionless) UDP?
how would things be different (in an addressing/security/restrictions sense) if I chose to use TCP instead?
I'm confident that there is a viable solution using two sockets at each end (one for A -> B data, one for B ->A data)... but is there a viable solution using a single socket at each end? Would there be any clear advantages to using one full-duplex socket per process if it is possible?
The question arises from a misunderstanding. The misunderstanding arises from reading variable names like receivePort and sendPort with different values, and reading them as if they have an implicit link to the socket at the local end. This might make one (mistakenly) believe that two sockets are being used, or must be used - one for send, one for receive. This is wrong - a single socket (at each end) is all that is required.
If using variables to refer to ports on a single host, it is preferable to name them such that it is clear that one is local or pertaining to "this" process, and the other is remote or peer and pertains to the address of a different process, despite being on the same local host. Then it should be clearer that, like any socket, it is entirely possibly to support both send and receive from the single socket with its single port number.
In this scenario (inter-process communication on the same host necessarily using different port numbers for the single socket at each end) all the other questions (SO_REUSEADDR, TCP vs UDP and receiving one's own transmissions) are distractions arising from the misunderstanding.
SCTP send single file using multiple stream and TCP send single file using single stream.
now question is "
how SCTP is better then TCP ?
" (in traffic scenario)
SCTP is not "better" than TCP in any way, but it does something different.
TCP emulates a reliable, ordered stream of octets over an unreliable unordered packet transport, which is conceptually very similar to reading from a file (without the ability to seek).
SCTP emulates a reliable in order distinct message delivery system (where "message" means as much as a defined chunk of data of some known length). Like UDP, it delivers one complete message at a time. Like TCP, it guarantees that messages arrive, and in the relative order in which they were sent.
SCTP has the ability to send different messages in separate streams, which allows to reduce latency, prevents head-of-line blocks, and makes better use of the available bandwidth in some scenarios. A web page with style information and images is the classic examples.
It does not send a single file via multiple streams (which would not make any sense).
(There are a few other features that I'm not naming because they have little relevance to the question)
SCTP can be considered as mix of UDP and TCP because it is message based like UDP and connection oriented like TCP to ensure in sequence delivery of messages along with congestion control mechanism. That is, SCTP is connection oriented but operates at message level.
It involves bundling of several connections into a single SCTP association operating on messages or chunks rather than bytes. This capability of SCTP to transmit several independent streams of chunks in parallel is referred to as Multistreaming which avoids head of line blocking. That is, in case of TCP, even though 3rd & 4th packet are fine, but if 2nd packet was lost, TCP shall do retransmission due to which 3rd and 4th packet has to wait until the 2nd packet is successfully/correctly received. However, incase of SCTP, this head of line blocking is reduced as single association is split into multiple independent streams of chunks(messages).
Also, note that SCTP facilitates to send messages in unordered mode too, which can completely avoid the head of line blocking wherein the upper layers should have mechanism for reordering of messages if required.
Ok folks, NOT counting ethernet speed (Infinitband), kernel bypass or any other fancy stuff, just plain TCP/IP (TCP/UDP over Ethernet) networking. What is the fastest messaging queue implementation that can deliver a message from host A to host B?
Let's assume 10Gigabits ethernet cards connecting both machines with up-to-date architecture and CPUs. What latency in microseconds are we talking here for a 1472 bytes message (MTU - IP/UDP headers)?
As #Sachin described very well, what I am looking for is the messaging queue and the latency number to send a message from A to B like below:
Host A <-------TCP-------> Messaging queue (process, route, etc) <-------TCP-------> Host B
if you do not require a broker in between, 0MQ gave us the best performance (you will need to test the numbers on your platform/use case). If using a broker in between, both ActiveMQ & RabbitMQ performed in the same range. Using Redis as a messaging server did not hold up for us.
If not using a messaging server, options such as Netty, J-groups etc might be useful (not sure about your programming language).
You could look into reliable UDP as well if going with straight socket connectivity.
hope it helps.
The lower bound would be at least 2 TCP connections and the routing time inside the messaging queue server (meaning the delays associated with these)
Host A <-------TCP-------> Messaging queue (process, route, etc) <-------TCP-------> Host B
Off course, if you build in redundancy, fault tolerance etc, then you are going to be certainly way above this lower bound.
It looks like you are talking about an UDP-based MQ because you mentioned MTU. Well, for UDP-based MQs this time is usually measured as the time required to publish a message and see it back in the message bus. So it is a round-trip time, not a one-way time as you described. This can usually be done in less than 6 microseconds, depending of course on your choice of LAN.