ZeroMQ push/pull pattern - push

I've decided to write a test code to see how pusher - many pullers bundle works and my suspicions came true.
Pullers receive messages in order they were connected, for example 1st message is received by 1st puller connected, 2nd by 2nd, etc. I've simulated a situation when one of the pullers stayed busy after receiving a message, but when it's time came to receive a message, it queued anyway, so I have 'lost' message. That's bad. I want this message to be received by next 'free' puller. Is that real?
My test code. I use zmqpp as bindings
void main()
{
auto _socket = sIpcContext->CreateNewSocket(zmqpp::socket_type::push);
_socket->bind("tcp://*:4242");
for (auto i = 0; i < 3; ++i)
{
new std::thread([&](int _idx)
{
auto idx = _idx;
auto sock = sIpcContext->CreateNewSocket(zmqpp::socket_type::pull);
sock->connect("tcp://127.0.0.1:4242");
for (;;)
{
std::string msg;
sock->receive(msg);
std::cout << idx << " received: " << msg << std::endl;
if (idx == 1)
{
std::cout << "Puller 1 is now busy" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(10000));
}
}
}, i);
}
for (auto i = 0;; ++i)
{
_socket->send(std::to_string(i));
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
I get this output:
0 received: 0
0 received: 1
1 received: 2
Puller 1 is now busy
2 received: 3
0 received: 4
2 received: 6
0 received: 7
2 received: 9
0 received: 10
2 received: 12
0 received: 13
2 received: 15
As you can see, 5, 8, and so on are 'missed' but actually queued in puller #1

Yes, push/pull sockets are dumb enough to let that happen. You could use other sockets, such as router/dealer, to send work to a free worker.

The 0MQ Guide explains this case (calls it the Post Office Analogy):
It's the post office analogy. If you have one queue per counter, and
you have some people buying stamps (a fast, simple transaction), and
some people opening new accounts (a very slow transaction), then you
will find stamp buyers getting unfairly stuck in queues. Just as in a
post office, if your messaging architecture is unfair, people will get
annoyed.
The solution in the post office is to create a single queue so that
even if one or two counters get stuck with slow work, other counters
will continue to serve clients on a first-come, first-serve basis.
Long story short, you should be using the ROUTER when dealing with slow running workers.

Related

One server and multiple clients in NS-3

I want to simulate a server that receives packets from multiple clients and process the data of these packets simultaneously together in NS-3. I have simulated one single server and client in NS-3 by modifying udp-echo-server and udp-echo-client applications in NS-3. Now, for implementing multiple clients, I modified the end lines of StartApplication function in udp-echo-server application as follows:
if((childpid=fork())==0)
{
m_socket->SetRecvCallback (MakeCallback(&UdpEchoServer::HandleRead, this));
m_socket6->SetRecvCallback (MakeCallback(&UdpEchoServer::HandleRead, this));
}
But it does not work. Actually, by connecting two clients, it just reads the first client and ignores the second client. It just runs StartApplication function once. Can anyone help me with this?
Thanks
The fundamental problem with what you're trying to do is that ns-3 is a single threaded simulator. You should not use fork to simulate forking. If you want multiple clients, you have to explicitly create them. I have quickly whipped up a simple example:
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
// simple udp multi-client, single-server simulation to answer
// https://stackoverflow.com/q/59632211/13040392
#include "ns3/core-module.h"
#include "ns3/internet-module.h"
#include "ns3/point-to-point-module.h"
#include "ns3/ipv4-global-routing-helper.h"
#include "ns3/applications-module.h"
#include "ns3/point-to-point-grid.h"
#include "ns3/flow-monitor-module.h"
using namespace ns3;
NS_LOG_COMPONENT_DEFINE("UdpMultiClient");
int
main(int argc, char *argv[]) {
// create grid structure of network
// not necessary. Could just create topology manually
PointToPointHelper p2pLink;
PointToPointGridHelper grid (2, 2, p2pLink);
InternetStackHelper stack;
grid.InstallStack(stack);
// assign IP addresses to NetDevices
grid.AssignIpv4Addresses (Ipv4AddressHelper ("10.1.1.0", "255.255.255.0"),
Ipv4AddressHelper ("10.2.1.0", "255.255.255.0"));
Ipv4GlobalRoutingHelper::PopulateRoutingTables();
// configure and install server app
int serverPort = 8080;
UdpEchoServerHelper serverApp (serverPort);
serverApp.Install(grid.GetNode(0,0));
Address serverAddress = InetSocketAddress(grid.GetIpv4Address(0,0), serverPort);
// configure and install client apps
UdpEchoClientHelper clientApp (serverAddress);
clientApp.Install(grid.GetNode(0,1));
clientApp.Install(grid.GetNode(1,0));
clientApp.Install(grid.GetNode(1,1));
// install FlowMonitor to collect simulation statistics
FlowMonitorHelper flowHelper;
Ptr<FlowMonitor> flowMonitor = flowHelper.InstallAll();
// configure and run simulation
Simulator::Stop(Seconds(10));
NS_LOG_UNCOND("Starting simulation.");
Simulator::Run();
Simulator::Destroy();
NS_LOG_UNCOND("Simulation completed.");
// simulation complete
// get statistics of simlation from FlowMonitor
flowMonitor->CheckForLostPackets();
std::map<FlowId, FlowMonitor::FlowStats> stats = flowMonitor->GetFlowStats();
uint64_t txPacketsum = 0;
uint64_t rxPacketsum = 0;
uint64_t DropPacketsum = 0;
uint64_t LostPacketsum = 0;
double Delaysum = 0;
for (std::map<FlowId, FlowMonitor::FlowStats>::const_iterator i = stats.begin(); i != stats.end(); ++i) {
txPacketsum += i->second.txPackets;
rxPacketsum += i->second.rxPackets;
LostPacketsum += i->second.lostPackets;
DropPacketsum += i->second.packetsDropped.size();
Delaysum += i->second.delaySum.GetSeconds();
}
NS_LOG_UNCOND(std::endl << " SIMULATION STATISTICS");
NS_LOG_UNCOND(" All Tx Packets: " << txPacketsum);
NS_LOG_UNCOND(" All Rx Packets: " << rxPacketsum);
NS_LOG_UNCOND(" All Delay: " << Delaysum / txPacketsum);
NS_LOG_UNCOND(" All Lost Packets: " << LostPacketsum);
NS_LOG_UNCOND(" All Drop Packets: " << DropPacketsum);
NS_LOG_UNCOND(" Packets Delivery Ratio: " << ((rxPacketsum * 100) / txPacketsum) << "%");
NS_LOG_UNCOND(" Packets Lost Ratio: " << ((LostPacketsum * 100) / txPacketsum) << "%");
// flowMonitor->SerializeToXmlFile("test.xml", true, true);
return 0;
}
As a quick note, in
UdpEchoClientHelper clientApp (serverAddress);
clientApp.Install(grid.GetNode(0,1));
clientApp.Install(grid.GetNode(1,0));
clientApp.Install(grid.GetNode(1,1));
we installed the UdpEchoClient on three Nodes. According to the documentation for this Application, UdpEchoClient sends a packet every 1000000000 ns = 1 s by default. Since we set the length of the simulation to 10 seconds using Simulator::Stop(Seconds(10));, we expect that each client will send 10 packets to the server. So, a total of 30 packets should be sent by clients. Also, since we are using UdpEchoServerHelper on the server, each packet will be echoed back by the server. Therefore, a total of 30 x 2 = 60 packets should be transmitted on the network.
The output of the script is
Starting simulation.
Simulation completed.
SIMULATION STATISTICS
All Tx Packets: 60
All Rx Packets: 60
All Delay: 0.0423177
All Lost Packets: 0
All Drop Packets: 0
Packets Delivery Ratio: 100%
Packets Lost Ratio: 0%
This answer actually demonstrates several features of ns-3, so feel free to ask any followup questions. I highly encourage you to check out the ns-3 documentation for classes you haven't encountered yet.

Read UART transmision Input buffer in Matlab

I'm trying to make a serial communication between two ESP8266 Wifi chips.
To start, I tried sending a sample data 10 times in a for loop. Here is the code:
Transmiter:
for Packets = 1 : 10
SendData(client,Data(Packets));
end
Receiver:
Packets = 1
while(1)
Data(Packets) = ReceiveData(Server);
Packets = Packets + 1;
if (packets == 10)
break
end
end
it works good. The problem is when I want to send data with some delays, the transmitter should connect to receiver again and the server (receiver) receives some data indicating that connection is made again.
The received Buffer should be:
+IPD,0,1024:ùüþþþýýþþÿÿûûýþýûúþÿúóýÿþþþþþýúøûýþ...
but after reconnecting the received Buffer is:
0,CLOSED %Receiver Prompt, disconnected from Transmiter
0,CONNECT %Receiver Prompt,connected to Transmiter
+IPD,0,1024:ùüþþþýýþþÿÿûûýþýûúþÿúóýÿþþþþþýúøûýþ...
The remaining part of data will be read in next packet and same for next packets.
what should I do to receive just the data?
The send and receive functions:
function ReceivedBuffer = ReceiveData(SerialPort)
ReceivedBuffer = fread(server,1038); %Size data = 1038 Bytes
end
function SendData(SerialPort,Data)
fwrite(SerialPort,Data);
end

VB6 client application which talks to one or multiple servers

I have a VB6 client application, which creates 1 or more (upto 4) sockets and connects to one or more TCP servers.
The client is supposed to continuously send requests to the server and wait for the server to respond for a certain responseTime. If the response does not arrive in the "responseTime", the client should send the next request on one of the sockets.
What is best way to make the client wait till the response arrives on the socket?
I do the following to have the client wait for the response/data to arrive: (Here the dataProcessed flag is set to True by the helper function invoked from the dataArrival() routine. This flag indicates that a response has been received and processed.
*Do While ((Timer < SentRequestTime) + responseTimeout) And (dataProcessed = False))
'DoEvents OR Sleep
Sleep 50
End If
Loop*
If I use "DoEvents" in the while loop, the application works fine for a while but later even though the response comes back to TCP layer (which I have examined through wireshark), the application does not get the DataArrival event.
If I use "sleep", the dataArrival event does not get delivered during the while loop, but arrives as soon as the loop is over. Using sleep makes the application non responsive.
What is the best way to have a single threaded VB6 socket client application to send a request, "wait for the data " to arrive for a certain time and then move on to the next request?
I would forget about both DoEvents() and Sleep() here. Those are tools of last resort, and nearly no program should contain either one. You need to "think 4th dimensionally" i.e. "Trust the Events, Luke!" This ain't your daddy's QBasic.
Here's a simulation where four Command buttons act as the servers, i.e. you click them manually as they become enabled. Two Timer controls are used here because we need to simulate processing time and transmission delay.
Option Explicit
'Use 4 Command buttons to simulate TCP sockets making server
'requests and getting back responses. Each "send" must get
'a response within RESPONSE_TIME_MS or be counted as a "miss."
'A new request is sent in either case.
Private Const PROCESS_TIME_MS As Long = 2000
Private Const PROCESS_TICKS As Long = PROCESS_TIME_MS \ 10
Private Const PROCESS_TICK_MS As Long = PROCESS_TIME_MS \ PROCESS_TICKS
Private Const RESPONSE_TIME_MS As Long = 4000
Private Const RESPONSE_TICKS As Long = RESPONSE_TIME_MS \ 10
Private Const RESPONSE_TICK_MS As Long = RESPONSE_TIME_MS \ RESPONSE_TICKS
Private ProcessCountdowns(0 To 3)
Private ResponseCountdowns(0 To 3)
Private Misses(0 To 3)
Private Sub SendRequest(ByVal Socket As Integer)
ResponseCountdowns(Socket) = RESPONSE_TICKS
cmdResponse(Socket).Enabled = True
End Sub
Private Sub cmdResponse_Click(Index As Integer)
'This is a "DataArrival" event.
'Process the response, then send a new request:
cmdResponse(Index).Enabled = False
ResponseCountdowns(Index) = 0
ProcessCountdowns(Index) = PROCESS_TICKS
End Sub
Private Sub Form_Load()
Dim Socket As Integer
For Socket = 0 To 3
SendRequest Socket
Next
tmrProcess.Interval = PROCESS_TICK_MS
tmrProcess.Enabled = True
tmrResponse.Interval = RESPONSE_TICK_MS
tmrResponse.Enabled = True
End Sub
Private Sub tmrProcess_Timer()
'This just simulates delay involved in processing responses and
'then sending new ones.
Dim Socket As Integer
For Socket = 0 To 3
If ProcessCountdowns(Socket) > 0 Then
ProcessCountdowns(Socket) = ProcessCountdowns(Socket) - 1
If ProcessCountdowns(Socket) <= 0 Then
SendRequest Socket
End If
End If
Next
End Sub
Private Sub tmrResponse_Timer()
Dim Socket As Integer
For Socket = 0 To 3
If ResponseCountdowns(Socket) > 0 Then
ResponseCountdowns(Socket) = ResponseCountdowns(Socket) - 1
If ResponseCountdowns(Socket) <= 0 Then
Misses(Socket) = Misses(Socket) + 1
lblMisses(Socket).Caption = CStr(Misses(Socket))
SendRequest Socket
End If
End If
Next
End Sub
Running the simulation requires two control arrays: one of 4 Command buttons and one of 4 Labels. Then it becomes a game of "Whack a Mole."
Pretty routine stuff actually, and the main reason we have Timer controls in the first place.

Best way to handle multiple connections at the same time

I have an application which listens to multiple connections and verifies whether the user is active or not
I use a 1 thread socket handling method with WSAASyncSelect.
The problem is that sometimes when a lot of users connecting at the same time some users get no reply
i think it is because the "send" hasn't been called yet and the program has received another connection so it goes again to handle the new connection ignoring the previous one. Like WSAASyncSelect has triggered and now it processing a new connection instead of completing the previous request.
So what to do to fix this issue? i tried to stop the events from WSAASyncSelect temporary by calling it with zero parameters when handling the connection until finish it then re enable network events but that didn't help either.
Here are the codes that handling the events (recieve then decrypt and then compare the bytes then send data according to what in listbox ie Active user or not)
This called upon receive of FD_READ
WSAAsyncSelect s, frmMain.hwnd, 0, 0 'Disabling Notifications event
Do Until bytesRecieved = SOCKET_ERROR
bytesRecieved = recv(wParam, buffer(Bytes), 500, 0)
If bytesRecieved > 0 Then
Bytes = Bytes + bytesRecieved
ElseIf bytesRecieved = 0 Then
Exit Sub
End If
Loop
Call MemCopy(ByVal decryptedArrival, buffer(0), Bytes)
WSAAsyncSelect s, frmMain.hwnd, WINSOCKMSG, FD_CONNECT + FD_READ + FD_CLOSE + FD_ACCEPT + FD_WRITE
If frmMain.chkSaveLog.value = vbChecked Then
frmMain.txtConnectionsLog.Text = frmMain.txtConnectionsLog.Text & Now & " Receiving a connection (" & wParam & ")" & vbNewLine
AutoScroll
If frmMain.chkAutoSave.value = vbChecked Then
strCurrentLogLine = Now & " Receiving a connection (" & wParam & ")"
AutoSaveLog (strCurrentLogLine)
frmMain.cmdClearLogs.Enabled = True
End If
End If
Below here is a decryption of bytes then comparing by ID as byte identifier like 1 = check for update
2 - send user info etc
in a Select Case statement following by a send Api.
And the accepting procedure
This called upon receive of FD_ACCEPT
Function AcceptConnection(wParam As Long)
lpString = String(32, 0)
AcSock = accept(wParam, sockaddress, Len(sockaddress))
strTempIP = getascip(sockaddress.sin_addr)
frmMain.txtConnectionsLog.Text = frmMain.txtConnectionsLog.Text & Now & " Getting a connection from IP address: " & _
strTempIP & " (" & AcSock & ")" & vbNewLine
AutoScroll
If frmMain.chkAutoSave.value = vbChecked Then
strCurrentLogLine = Now & " Getting a connection from IP address: " & strTempIP & " (" & AcSock & ")" & vbNewLine
AutoSaveLog (strCurrentLogLine)
End If
End Function
Are there any suggestions for a better performance?
What you showed is NOT the correct way to use WSAAsyncSelect(). Try something more like this instead:
When creating a listening socket:
lSock = socket(...)
bind(lSock, ...)
listen(lSock, ...)
WSAAsyncSelect lSock, frmMain.hwnd, WINSOCKMSG, FD_ACCEPT
When a listening socket receives FD_ACCEPT:
Function AcceptConnection(wParam As Long)
AcSock = accept(wParam, sockaddress, Len(sockaddress))
If AcSock = INVALID_SOCKET Then
Exit Sub
End If
WSAAsyncSelect AcSock, frmMain.hwnd, WINSOCKMSG, FD_READ + FD_CLOSE + FD_WRITE
...
End Function
When an accepted client socket receives FD_READ:
Function ReadConnection(wParam As Long)
Do
bytesRecieved = recv(wParam, ReadBuffer(ReadBytes), 500, 0)
If bytesRecieved = SOCKET_ERROR Then
If WSAGetLastError() <> WSAEWOULDBLOCK Then
Exit Sub
End If
ElseIf bytesRecieved = 0 Then
Exit Sub
Else
ReadBytes = ReadBytes + bytesRecieved
End If
Loop Until bytesRecieved = SOCKET_ERROR
' process ReadBuffer up to ReadBytes number of bytes as needed...
' remove processed bytes from front of ReadBuffer and decrement ReadBytes accordingly
...
End Function
When an accepted client socket receives FD_WRITE:
Function WriteConnection(wParam As Long)
While SendBytes > 0
bytesSent = send(wParam, SendBuffer(0), SendBytes, 0)
If bytesSent = SOCKET_ERROR Then
Exit Sub
End If
' remove bytesSent number of bytes from front of SendBuffer ...
SendBytes = SendBytes - bytesSent;
End While
End Function
The trick is that you need to allocate separate ReadBuffer and SendBuffer buffers for each accepted client. Make sure that each time you receive FD_READ that you are appending bytes only to the ReadBuffer of the socket that triggered FD_READ, and each time you receive FD_WRITE that you are removing bytes only from the SendBuffer of the socket that triggered FD_WRITE.
When recv() has no more bytes to read, process that socket's ReadBuffer as needed, removing only complete messages from the front and leaving incomplete messages for later processing.
When send() fails with WSAEWOULDBLOCK, append any unsent bytes to the SendBuffer of the socket that caused send() to fail. When you receive an FD_WRITE event for a socket, check that socket's SenBuffer and resend any bytes that are in it, stopping when the buffer is exhausted or an WSAEWOULDBLOCK error occurs.
Very easy, and quite effective, way to do it is to fork out for every incoming connection. This will most likely require you to restructure your application, but the basic flow should be as follows:
1. New connection is opened to the server
2. Server accepts the connection and forks out
3. The fork closes the original socket for listening, so only the parent will be accepting new connections
4. And then your magic happens, separate from the original thread.
This way you do not have to worry about issues of concurrency, as long as your machine can handle all the traffic and load because each connections is independent.

VB6 RS232 not receiving full data from device using MSCOMM Controll

I have a clinical device that sends data on com port, I want to receive data from device
it also received First Frame (254) character after send ACK on ENQ
it receive [ETB] [CR][LF] characters
then I again send ACK for next frame, but not receive data
only receiving EOT char
Device Communication as per device is:
<-[ENQ]
->[ACK]
<-[STX]1H|**********************-[ETB]21[CR][LF]
->[ACK]
<-[STX]1H|**********************-[ETX]8E[CR][LF]
->[ACK]
<-[EOT]
my code is:
'MSComm1.Settings = "9600,n,8,1"
'MSComm1.InputLen = 1
Private Sub MSComm1_OnComm()
Dim InBuff As String
InBuff = MSComm1.Input
if Chr$(5)=InBuff then 'ENQ received
MSComm1.Output=Chr$(6) & VbCr
elseif Chr$(10)=InBuff then 'LF received
MSComm1.Output=Chr$(6) & VbCr
else
text1.text=text1.text & InBuff
end if
End Sub
Device sending full data because 1 software comes with device which receive full data as
but I didn't receive next frame after send ACK again,
if any one have idea what output have to send FOR next ACK, please advice me
thanks in advance
Do something like this...
MSComm1.InputLen = 1 ' for sending single character from device
MSComm1.RThreshold = 1 ' for firing events on receiving a single character
Dim InBuff As String
if MSComm1.CommEvent = comEvReceive then
do
InBuff = MSComm1.Input
Loop Until MSComm1.InBufferCount < 1
Firstly receive all the data and after that use that in your own way.