I'm having trouble with OPC UA Subscriptions and Notifications in the ANSI C stack. OPC UA Part 4, Service says:
5.13.1 Subscription model
5.13.1.1 Description c) NotificationMessages are sent to the Client in response to Publish requests.
Sent how? I'm really expecting a callback of some sort, but there doesn't seem to be one. It does say these are in response to a 'Publish' request, but a Publish service call acknowledges receipt of a notification, it doesn't seem to request one. Besides, that would be polling and the whole point of Subscriptions and Monitoring is to not poll.
Can anyone supply an example showing monitoring of a data value in ANSI C?
PublishRequests are queued on the server and responses are only returned when notifications are ready or a keep-alive needs to be sent (or a bunch of other stuff, check the state machine description in part 4).
They do include acknowledgements of previously received notifications as well, but the idea is that the response isn't expected immediately and that the client will generally keep pumping PublishRequests out so that the server has a queue of them ready to return notifications whenever a subscription needs to.
Yes, it's polling. It's a bit of a bummer that it's not strictly unsolicited, but that's how it works.
__
edit:
It's not really polling. It's batched report by exception with a QoS guarantee and back pressure mechanism provided by subsequent PublishRequests.
This is C# code. I hope that it will help you.
private NotificationMessageReceivedEventHandler
m_NotificationMessageReceived;
// ...
m_NotificationMessageReceived =
new NotificationMessageReceivedEventHandler
(Subscription_NotificationMessageReceived);
m_subscription.NotificationMessageReceived +=
Subscription_NotificationMessageReceived;
// ...
private void Subscription_NotificationMessageReceived
(Subscription subscription,
NotificationMessageReceivedEventArgs e)
{
if (e.NotificationMessage.NotificationData == null ||
e.NotificationMessage.NotificationData.Count == 0)
{
LogMessage("{0:HH:mm:ss.fff}: KeepAlive",
e.NotificationMessage.PublishTime.ToLocalTime());
}
}
Related
I am new NATS. Not sure how NATS request reply works.
As per my understanding, this pattern can be use for bi-directional communication but questions is, Does it works between same message id/thread ? If not, can't we use two different queue for the same purpose? How it is different from pub-sub or queue pattern of NATS?
Can someone provide more use case on this?
Thanks.
You added nats-streaming-server tag, so I would first want to clarify that there is no request/reply API in NATS Streaming, because it does not really make sense.
In NATS, you would use request/reply when your publishing application wants to know that the subscribing application did receive and process the message. It is an end-to-end confirmation that the published message was received and processed.
It can also be simply because the subscribing application processes a job and send the result of that job back to the requestor.
A simple example would be:
// Request will create an internal subscription on
// a private inbox and set it to the message's Reply
// field.
msg, err := nc.Request("job", payload, time.Second)
if err != nil {
...
} else {
// msg is the reply sent by the subscribing application.
}
In the other side, you would have registered a subscription to handle the job requests.
nc.Subscribe("job", func(req *nats.Msg) {
// req is the request received by the publisher above.
// Send back a reply to the request reply subject.
nc.Publish(req.Reply, []byte(reply))
})
Not sure what language you use, but here is a link to the Go client
I use the perform javascript call to perform an action on the server, like this:
subscription.perform('action', {...});
However, from what I've seen there seems to be no builtin javascript "success" callback, i.e. to let me know the action is concluded on the server's side (or possibly failed). I was thinking about sending a broadcast at the end of the action like so:
def action(data)
...do_stuff
ActionCable.server.broadcast "room", success_message...
end
But all clients subscribed to this "room" would receive that message, possibly resulting in false positives. In addition, from what I've heard, message order isn't guaranteed, so a previous broadcast inside this action could be delivered after the success message, possibly leading to further issues.
Any ideas on this or am I missing something completely?
Looking at https://github.com/xtian/action-cable-js/blob/master/dist/cable.js and , https://developer.mozilla.org/en-US/docs/Web/API/WebSocket#send(), perform just executes WebSocket.send() and returns true or false, and there is no way to know whether your data has arrived. (That is just not possible with WebSockets, it seems.)
You could try using just a http call (I recommend setting up an api with jbuilder), or indeed broadcasting back a success message.
You can solve the order of the messages by creating a timestamp on the server, and sending it along with the message, and then sorting the messages with Javascript.
Good luck!
Maybe what you are looking for is the trasmit method: https://api.rubyonrails.org/v6.1.3/classes/ActionCable/Channel/Base.html#method-i-transmit
It sends a message to the current connection being handled for a channel.
Background
We're building an application that will process & send emails via Mailgun. These are sometimes one-off messages, initiated by a transaction. Some emails, though, will be sent to 30k+ at once.
Eg, a newsletter to all members.
Considerations
Mailgun offers a Batch Sending option with their API. Using "Recipient Variables", you can include dynamic values that are paired with a particular user.
This Batch Sending functionality is limited, however. You cannot send more than 1,000 recipients per request, which means we have to iterate through a recipient list (on our database) for each set of 1,000. Mailgun provides an example of how this might work, using Python (scroll about 2/3 down).
Question
Are there any advantages to batch sending (ie, sending an email to a group of recipients through a single API call, using recipient variables) as opposed to making our own loop, variable substitutions and individual API calls?
I assume this is more taxing on our server, as it would be processing each message itself, instead of just offloading all that data to Mailgun's server for heavy-lifting on their end. But I also like the flexibility & simplicity of handling that on our end and sending a "fully-rendered" message to Mailgun, one at a time, without having to iterate 1k at a time.
Any thoughts on best practices, or considerations we should take into account?
Stumbled onto this today, and felt it provided a pretty good summary/answer for my original question. I wanted to post this as an answer, in case anybody else has this question and hasn't found this Mailgun post. Straight from the horse's mouth, too. The nutshell version:
For PHP, at least, the SDK has a Mailgun class, with a BatchMessage() method. This actually handles the counting of recipients for you, so you can just queue up as many email addresses as you want (ie, more than 1k) and Mailgun will fire off to the API endpoint as needed. Pretty slick!
Here's their original wording, plus a link to the page.
Sending a message with Mailgun PHP SDK + Batch Message:
Batch Message
In addition to Message Builder, we have Batch Message. This class
allows you to build a message and submit a template message in
batches, up to 1,000 recipients per post. The benefit of using this
class is that the recipients tally is monitored and will automatically
submit the message to the endpoint when you've added the 1,000th
recipient. This means you can build your message and begin iterating
through your database. Forget about sending the message, the SDK will
keep track of posting to the API when necessary.
// First, instantiate the SDK with your API credentials and define your domain.
$mgClient = new Mailgun("key-example");
$domain = "example.com";
// Next, instantiate a Message Builder object from the SDK, pass in your sending domain.
$batchMsg = $mgClient->BatchMessage($domain);
// Define the from address.
$batchMsg->setFromAddress("dwight#example.com",
array("first"=>"Dwight", "last" => "Schrute"));
// Define the subject.
$batchMsg->setSubject("Help!");
// Define the body of the message.
$batchMsg->setTextBody("The printer is on fire!");
// Next, let's add a few recipients to the batch job.
$batchMsg->addToRecipient("pam#example.com",
array("first" => "pam", "last" => "Beesly"));
$batchMsg->addToRecipient("jim#example.com",
array("first" => "Jim", "last" => "Halpert"));
$batchMsg->addToRecipient("andy#example.com",
array("first" => "Andy", "last" => "Bernard"));
// ...etc...etc...
// After 1,000 recipeints,
// Batch Message will automatically post your message to the messages endpoint.
// Call finalize() to send any remaining recipients still in the buffer.
$batchMsg->finalize();
The answer of #cdwyer and #nikoshr is very helpful, but bit legacy. Used methods in the example are deprecated. Here is current usage of lib:
$batchMessage = $this->mailgun->messages()->getBatchMessage('mydomain.com');
$batchMessage->setFromAddress('user#domain.com');
$batchMessage->setReplyToAddress('user2#domain.com');
$batchMessage->setSubject('Contact form | Company');
$batchMessage->setHtmlBody('<html>...</html>');
foreach ($recipients as $recipient) {
$batchMessage->addToRecipient($recipient);
}
$batchMessage->finalize();
More info at documentation.
I noticed in the FAQ, in the Monitoring section, that it's not possible to get a list of connected peers or to be notified when peers connect/disconnect.
Does this imply that it's also not possible to know which topics a PUB/XPUB socket knows it should publish, from its upstream feedback? Or is there some way to access that data?
I know that ZMQ >= 3.0 "supports PUB/SUB filtering at the publisher", but what I really want is to filter at my application code, using the knowledge ZMQ has about which topics are subscribed to.
My use-case is that I want to publish info about the status of a robot. Some topics involve major hardware actions, like switching the select lines on an ADC to read IR values.
I have a publisher thread running on the bot that should only do that "read" to get IR data when there are actually subscribers. However, since I can only feed a string into my pub_sock.send, I always have to do the costly operation, even if ZMQ is about to drop that message when there are no subscribers.
I have an implementation that uses a backchannel REQ/REP socket to send topic information, which my app can check in its publish loop, thereby only collecting data that needs to be collected. This seems very inelegant though, since ZMQ must already have the data I need, as evidenced by its filtering at the publisher.
I noticed that in this mailing list message, the OP seems to be able to see subscribe messages being sent to an XPUB socket.
However, there's no mention of how they did that, and I'm not seeing any such ability in the docs (still looking). Maybe they were just using Wireshark (to see upstream subscribe messages to an XPUB socket).
Using zmq.XPUB socket type, there is a way to detect new and leaving subscribers. The following code sample shows how:
# Publisher side
import zmq
ctx = zmq.Context.instance()
xpub_socket = ctx.socket(zmq.XPUB)
xpub_socket.bind("tcp://*:%d" % port_nr)
poller = zmq.Poller()
poller.register(xpub_socket)
events = dict(poller.poll(1000))
if xpub_socket in events:
msg = xpub_socket.recv()
if msg[0] == b'\x01':
topic = msg[1:]
print "Topic '%s': new subscriber" % topic
elif msg[0] == b'\x00':
topic = msg[1:]
print "Topic '%s': subscriber left" % topic
Note that the zmq.XSUB socket type does not subscribe in the same manner as the "normal" zmq.SUB. Code sample:
# Subscriber side
import zmq
ctx = zmq.Context.instance()
# Subscribing of zmq.SUB socket
sub_socket = ctx.socket(zmq.SUB)
sub_socket.setsockopt(zmq.SUBSCRIBE, "sometopic") # OK
sub_socket.connect("tcp://localhost:%d" % port_nr)
# Subscribing zmq.XSUB socket
xsub_socket = ctx.socket(zmq.XSUB)
xsub_socket.connect("tcp://localhost:%d" % port_nr)
# xsub_socket.setsockopt(zmq.SUBSCRIBE, "sometopic") # NOK, raises zmq.error.ZMQError: Invalid argument
xsub_socket.send_multipart([b'\x01', b'sometopic']) # OK, triggers the subscribe event on the publisher
I'd also like to point out the zmq.XPUB_VERBOSE socket option. If set, all subscription events are received on the socket. If not set, duplicate subscriptions are filtered. See also the following post: ZMQ: No subscription message on XPUB socket for multiple subscribers (Last Value Caching pattern)
At least for the XPUB/XSUB socket case you can save a subscription state by forwarding and handling the packages manually:
context = zmq.Context()
xsub_socket = context.socket(zmq.XSUB)
xsub_socket.bind('tcp://*:10000')
xpub_socket = context.socket(zmq.XPUB)
xpub_socket.bind('tcp://*:10001')
poller = zmq.Poller()
poller.register(xpub_socket, zmq.POLLIN)
poller.register(xsub_socket, zmq.POLLIN)
while True:
try:
events = dict(poller.poll(1000))
except KeyboardInterrupt:
break
if xpub_socket in events:
message = xpub_socket.recv_multipart()
# HERE goes some subscription handle code which inspects
# message
xsub_socket.send_multipart(message)
if xsub_socket in events:
message = xsub_socket.recv_multipart()
xpub_socket.send_multipart(message)
(this is Python code but I guess C/C++ looks quite similar)
I'm currently working on this topic and I will add more information as soon as possible.
I've been having some problems with the below code that I've pieced together. All the events work as advertised however, when a client drops off-line without first disconnecting the close event doesn't get call right away. If you give it a minute or so it will eventually get called. Also, I find if I continue to send data to the client it picks up a close event faster but never right away. Lastly, if the client gracefully disconnects, the end event is called just fine.
I understand this is related to the other listen events like upgrade and ondata.
I should also state that the client is an embedded device.
client http request:
GET /demo HTTP/1.1\r\n
Host: example.com\r\n
Upgrade: Websocket\r\n
Connection: Upgrade\r\n\r\n
//nodejs server (I'm using version 6.6)
var http = require('http');
var net = require('net');
var sys = require("util");
var srv = http.createServer(function (req, res){
});
srv.on('upgrade', function(req, socket, upgradeHead) {
socket.write('HTTP/1.1 101 Web Socket Protocol Handshake\r\n' +
'Upgrade: WebSocket\r\n' +
'Connection: Upgrade\r\n' +
'\r\n\r\n');
sys.puts('upgraded');
socket.ondata = function(data, start, end) {
socket.write(data.toString('utf8', start, end), 'utf8'); // echo back
};
socket.addListener('end', function () {
sys.puts('end'); //works fine
});
socket.addListener('close', function () {
sys.puts('close'); //eventually gets here
});
});
srv.listen(3400);
Can anyone suggest a solution to pickup an immediate close event? I am trying to keep this simple without use of modules. Thanks in advance.
close event will be called once TCP socket connection is closed by one or another end with few complications of rare cases when system "not realising" that socket been already closed, but this are rare cases. As WebSockets start from HTTP request server might just keep-alive till it timeouts the socket. That involves the delay.
In your case you are trying to perform handshake and then send data back and forth, but WebSockets are a bit more complex process than that.
The handshake process requires some security procedure to validate both ends (server and client) and it is HTTP compatible headers. But different draft versions supported by different platforms and browsers do implement it in a different manner so your implementation should take this in account as well and follow official documentation on WebSockets specification based on versions you need to support.
Then sending and receiving data via WebSockets is not pure string. Actual data sent over WebSockets protocol has data-framing layer, which involves adding header to each message you send. This header has details over message you sending, masking (from client to server), length and many other things. data-framing depends on version of WebSockets again, so implementations will vary slightly.
I would encourage to use existing libraries as they already implement everything you need in nice and clean manner, and have been used extensively across commercial projects.
As your client is embedded platform, and server I assume is node.js as well, it is easy to use same library on both ends.
Best suit here would be ws - actual pure WebSockets.
Socket.IO is not good for your case, as it is much more complex and heavy library that has multiple list of protocols support with fallbacks and have some abstraction that might be not what you are looking for.