can anyone tell me exactly how interactions between bitcoin nodes happens?
I mean i need the order of messages the nodes send and receive.
For example i know that the first message that two nodes exchange is "version" and "verack" messages but often between "version" and "verack" they send many other messages like "sendcmpct", "getheader"... etc, but other times they only send "version" and "verack" before any other messages.
Maybe this article can help (check all Key points)
https://nownodes.io/blog/how-bitcoin-works/
Related
Suppose I have the following supply chain model see model model1
Agents are communicating with each other through a defined network and send messages to each other through ports. for example, demand is generated for customers through their ports and send as "orders" upstream to facilities. Upstream facilities send "shipments" to downstream facilities
and stats are collected at each node.
The model seems to work for 2 echelons but when one facility is connected to two facilities downstream as desired I get the following error "Agent can't be in several flowcharts at the time. At least two flowchart blocks are in conflict" see error. Based on the description it seems the agent "shipment" is sent to two facilities at the same time.
My question is how could I avoid this conflict?
more information about each node:
Agents' "orders" enter through each node's port and are capture as Enter. take(msg), follow a flowchart, and exit as Agent "shipment" to each destination. Each agent "order" has a double amount and port destination. see facility node
any suggestions please?
You must make sure that you do not send agents into a flowchart that is already in another flow chart, correct. This is bad model design.
One way to debug and find the root issue: before sending any message agent, check currentBlock()!=null and traceln the agent and the block. Also pause the model.
You can then see where you want to (re)send that agent that is already in some other flowchart block.
You probably send message agents out that are still somewhere else.
PS: For messages, you probably do not want to use flow charts at all but normal message passing. This avoids these pains here as you can easily send the same message to several agents. Check how message passing is done in the example agent models
In a kerberized set-up with a Kafka cluster I can easily enforce that for topic X, Users A and B have the right to produce messages, and for topic Y, User C has the right to produce messages.
In terms of traceability, this gives me the guarantee that all messages on topic Y are produced by User C. However, I would also like to know which messages on topic X were produced by User A, and which messages were produced by User B.
Is there an option to tag each message with the user that produced it? With one remark perhaps, that I do not want to do this on the producer-side but would like to enforce it broker-side so I have an absolute guarantee that this user information is present.
Any help would be greatly appreciated.
I don't think there currently is a way to do that to be honest. The only two pluggable classes in the broker that I know of are the KafkaPrincipalBuilder and the Authorizer, both of which have the principal you are after, but do not have access to the actual records to add the principal in a header field or wherever one would want it.
And even if we look to the producer side, the two options there would be Interceptor and Serializer classes that you can customize, both of which have access to the records, but not the user principal.
If you actually want to enforce this I'm afraid you'll have to become a bit creative and do something like have them sign every message with a private key in a header field or something similar - when processing data on the other end you can then discard anything that doesn't contain a that field.
Apart from that I think there is not much you can do except actually changing Kafka itself to add this information - in general the handleProduceRequest method should be a good starting place, here you have the requests that have been authorized and can also access the requests session which contains the principal. Records are only represented as bytebuffers here though, not sure whether you can easily headers at this point tbh..
The scenario is publisher/subscriber, and I am looking for a solution which can give the feasibility of sending one message generated by ONE producer across MULTIPLE consumers in real-time. the light weight this scenario can be handled by one solution, the better!
In case of AMQP servers I've only checked out Rabbitmq and using rabbitmq server for pub/sub pattern each consumer should declare an anonymous, private queue and bind it to an fanout exchange, so in case of thousand users consuming one message in real-time there will be thousands or so anonymous queue handling by rabbitmq.
But I really do not like the approach by the rabbitmq, It would be ideal if rabbitmq could handle this pub/sub scenario with one queue, one message , many consumers listening on one queue!
what I want to ask is which AMQP server or other type of solutions (anyone similar including XMPP servers or Apache Kafka or ...) handles the pub/sub pattern/scenario better and much more efficient than RabbitMQ with consuming (of course) less server resource?
preferences in order of interest:
in case of AMQP enabled server handling the pub/sub scenario with only ONE or LESS number of queues (as explained)
handling thousands of consumers in a light-weight manner, consuming less server resource comparing to other solutions in pub/sub pattern
clustering, tolerating failing of nodes
Many Language Bindings ( Python and Java at least)
easy to use and administer
I know my question may be VERY general but I like to hear the ideas and suggestions for the pub/sub case.
thanks.
In general, for RabbitMQ, if you put the user in the routing key, you should be able to use a single exchange and then a small number of queues (even a single one if you wanted, but you could divide them up by server or similar if that makes sense given your setup).
If you don't need guaranteed order (as one would for, say, guaranteeing that FK constraints wouldn't get hit for a sequence of changes to various SQL database tables), then there's no reason you can't have a bunch of consumers drawing from a single queue.
If you want a broadcast-message type of scenario, then that could perhaps be handled a bit differently. Instead of the single user in the routing key, which you could use for non-broadcast-type messages, have a special user type, say, __broadcast__, that no user could actually have, and have the users to broadcast to stored in the payload of the message along with the message itself.
Your message processing code could then take care of depositing that message in the database (or whatever the end destination is) across all of those users.
Edit in response to comment from OP:
So the routing key might look something like this message.[user] where [user] could be the actual user if it were a point-to-point message, and a special __broadcast__ user (or similar user name that an actual user would not be allowed to register) which would indicate a broadcast style message.
You could then place the users to which the message should be delivered in the payload of the message, and then that message content (which would also be in the payload) could be delivered to each user. The mechanism for doing that would depend on what your end destination is. i.e. do the messages end up getting stored in Postgres, or Mongo DB or similar?
I'm very new to messaging system, and I was trying to find my answer on http://www.rabbitmq.com/tutorials/, and I'm pretty sure it should be over there, but so far I got little bit confused with all bindings, queues, exchanges.
So I'm looking an answer for the question how to specify what type of "queue" (sorry if i have to use other word for this) is it. On producer side. To be more clear I'll give you an example:
So I want my consumer to subscribe to one "queue" and than once it receives it perform some operation based on what's inside this queue. Lets say if message contains a picture than do something, if it is a text, than do something else.
I was thinking my producer should add something like type:foo to the payload, and than consumer will look for this type. But I hope there is a better solution for this. Something like add a header to the queue.
Thank you.
If your consumer have to do different tasks for different types of message, then it would be better to create one distinct consumer per task.
That way, you can easily create one queue for each type of message and make each consumer consume messages from the right queue.
Your producer can send the message to the correct queue either directly or by using RabbitMQ routing.
Take a look at the "Routing" tutorial on the RabbitMQ website, it seems to match your use-case : http://www.rabbitmq.com/tutorials/tutorial-four-python.html
I have been working with qpid and now i am trying to move to broker less messaging system , but I am really confused about network traffic in a Pub Sub pattern. I read the following document :
http://www.250bpm.com/pubsub#toc4
and am really confused how subscription forwarding is actually done ?
I thought zero mq has to be agnostic for the underlying network topology but it seems it is not. How does every node knows what to forward and what to not (for e.g. : in eth network , where there can be millions subscriber and publisher , message tree does not sound a feasible to me . What about the hops that do not even know about the existence of zero mq , how would they forward packets to subscribers connected to them , for them it would be just a normal packet , so they would just forward multiple copies of data packets even if its the same packet ?
I am not networking expert so may be I am missing something obvious about message tree and how it is even created ?
Could you please give certain example cases how this distribution tree is created and exactly which nodes are xpub and xsub sockets created ?
Is device (term used in the link) something like a broker , in the whole article it seemed like device is just any general intermediary hop which does not know anything about zero mq sockets (just a random network hop) , if it is indeed a broker kind of thing , does that mean for pub sub , all nodes in messaging tree have to satisfy the definition of being a device and hence it is not a broke less design ?
Also in the tree diagram (from the link , which consist P,D,C) , I initially assumed C and C are two subscribers and P the only publisher (D just random hop), but now it seems that we have D as the zero mq . Does C subscribes to D and D subscribes to P ? or both the C just subscribe to P (To be more generic , does each node subscribe to its parent only in the ). Sorry for the novice question but it seems i am missing on something obvious here, it would be nice if some one can give more insights.
zeromq uses the network to establish a connection between nodes directly (e.g via tcp), but only ever between 1 sender and 1-n receivers. These are connected "directly" and can exchange messages using the underlying protocol.
Now when you subscribe to only certain events in a pub-sub scenario, zeromq used to filter out messages subscriber side causing unnecessary network traffic from the publisher to at least a number of subscribers.
In newer versions of zeromq (3.0 and 3.1) the subscriber process sends its subscription list to the publisher, which manages a list of subscribers and the topics they are interested in. Thus the publisher can discard messages that are not subscribed too by any subscriber and potentially send targeted messages at only interested subscribers.
When the publisher is itself a subscriber of events (e.g. a forwarding or routing device service) it might forward those subscriptions again by similarly subscribing to its connected publishers.
I am not sure whether zeromq still does client side filtering in newer versions even if it "forwards" its subscriptions though.
A more efficient mechanism for pub/sub to multiple subscribers is to use multicast whereby a single message traverses the network and is received by all subscribers (who can then filter what they wish).
ZeroMQ supports a standardised reliable multicast called Pragmatic General Multicast.
These references should give you an idea how it all works. Note that multicast generally only works on a single subLAN and may need router configuration or TCP bridges to span multiple subLANs.