It is very well-known that Vector clocks detect causal dependencies between events, but there are different implementation methods.
What are the differences (advantages/disadvantages) between:
ticking only when sending (before).
ticking only when sending (after).
ticking when receiving and ticking before sending
ticking when receiving and ticking after sending
It is suppose to tick right before sending and also tick right before receiving. The word right before here means the event represented send/receive will have the vector clock timestamp of right after the vector clock has just been updated with a tick.
The reason is because there are 2 processes, the sender process and the receiver process.
It needs to tick the sending event so that the sender process is aware that a send event has just occurred. Correspondingly, it also needs to tick in the receiving event so that the receiver process is aware that a receive event has just occurred.
Otherwise, it would violate Lamport's happened before relation in either sender or receiver if a tick did not occur there.
Related
I understand that the operation of a CANopen inhibit timer is to ensure a minimum time between successive transmissions of the same message, but the specification does not make it clear what to do if the data changes during the inhibit time (and the transmission is on change-of-state). Should I buffer the data and transmit it when the inhibit timer expires, or discard it and wait for a change after the timer has expired?
My assumption would be, since it is not clearly defined, I can choose whichever approach I want, but I'd appreciate the input of any experienced architects / developers on this.
Thanks.
You're correct that the inhibit time is simply the minimum time between consecutive CAN frames with the same CAN-ID. The standard does not specify the behavior for multiple events during the inhibit time window, because it depends on the situation.
For services like NMT, EMCY and perhaps LSS, you'd want to buffer the messages and send them later. In this case the inhibit time is simply a means to help slow (or badly programmed) devices to handle short bursts of messages. I've seen devices that could only handle 3 CAN frames at once, so it's often necessary, but you would not want them to miss messages.
For event-driven Transmit-PDOs, it depends on what the PDO represents. If you use it to track state, it might make sense to drop events during the inhibit window. They're invalidated by subsequent events anyway. To ensure you always emit the latest state, you can store the most recent event and transmit it once the inhibit time has elapsed, or use the event-timer to ensure you're never too far behind. I've used this strategy in the past for analog inputs where line noise would sometimes cause event bursts.
If you use PDOs to track events (or state changes), you'd be better of buffering them so no events get lost. However, this can introduce potentially unbounded delays if the event period is shorter than the inhibit time.
For the products we're working on at Lely (dairy farm robots), we actually prefer to use SYNC-driven PDOs instead. It results in a much more predictable CAN bus load. And we don't have to track state at the receiver side because we receive a full update on every SYNC. However, the receiver is always one SYNC period behind the transmitter, so this may not be appropriate for your use case.
Any idea how to make kafka-to-kafka mirroring but with a sampling (for example only 10% of the messages)?
You could use MirrorMakerMessageHandler (which is configured by message.handler parameter):
https://github.com/apache/kafka/blob/1.0/core/src/main/scala/kafka/tools/MirrorMaker.scala#L430
The handler itself would need to make a decision whether to forward a message. A simple implementation would be just a counter of messages received, and forwarding if 0 == counter % 10.
However this handler is invoked for every message received, so it means that you'd be receiving all of messages & throwing away 90% of them.
The alternative is to modify main loop, where the mirror maker consumer receives the message, and forwards it to producers (that send the message to mirror cluster) is here
https://github.com/apache/kafka/blob/1.0/core/src/main/scala/kafka/tools/MirrorMaker.scala#L428
You would need to modify the consumer part to either-or:
forward only N-th (10th) message/offset
seek to only N-th message in log
I prefer the former idea, as in case of multiple MM instances in the same consumer group, you would still get reasonable behaviour. Second choice would demand more work from you to handle reassignments.
Also, telling which message is from 10% is non-trivial, I just assumed that it's every 10th message received.
I have a ZMQ_PUB socket sending messages out at ~50Hz. One destination needs to react to each message, so it has a standard ZMQ_SUB socket with a while(true) loop checking for new messages. A second destination should only react once a second to the "most recent" message. That is, my second destination needs to subsample.
For the second destination, I believe I'd want to have a time-based loop that is called at my desired rate (1Hz) and recv() the latest message, dropping the rest. I believe this is done via a ZMQ_HWM on the subscriber. Is there another option that needs to be set somewhere?
Do I need to worry about the different subscribers having different HWMs? Will the publisher become angry? It's a shame ZMQ_RATE only applies to multicast sockets.
Is there a best way to accomplish what I'm attempting?
zmq v3.2.4
The HighWaterMark will not be a fantastic solution for your problem. Setting it on the subscriber to, let's say, 10 and reading 1 message per second, will just give you the old messages first, slowly, and throw away all the new, because it's limit are reached.
You could either use a topic on you publisher that makes you able to filter out every 50th message like making the topic messageCount % 50 and subscribe to 0.
Otherwise maybe you shouldn't use zmq's pub/sub, but instead do you own look alike with router/dealer that allows you to subscribe to sampled messages.
Lastly you could also just send them all. 50 m/s is hardly anything in zmq (if they aren't heavy on data, like megs) and then only use every 50th message.
I'm reading a MIDI file and I'm having trouble determining when next events trigger.
Let's say I have a midi file that has a track like this (where T=n is the delta time):
[T=0: Note On, C4] [T=128: Note Off, C4] [T=0: Note On, D4] [T=128: Note Off, D4]
Does the second Note On (D4) take place at the EXACT same time/tick as the previous Note Off (C4)? Or do you trigger it on the next tick?
In theory, the two events happen at the same time.
In practice, events need a certain time to be sent over MIDI (about one millisecond for three bytes), but the second event will be sent as soon as possible after the first one.
When no actual MIDI cable is involved, the events actually could take effect at the same time.
All events happen on a tick. However, they're sent out over the MIDI cable one at a time since MIDI is both a serial protocol and serial hardware. This became a problem with devices that sent out huge numbers of controller change messages, originally like the MIDI guitar controllers. They simply sent out more MIDI messages per second than the cable could transmit.
On alternate transport, like USB, those events can happen closer together but because they are serial, they must still happen one after the other. That time frame may be indistiguishable, (we hope), but there will always be a tiny lag.
For them to happen at the "same" time, you must either a) buffer or b) make them happen in different places, as with parallel players, which still leaves you with a delay in syncing.
I am trying to implement a poker server. An http server forwards data packets to the backend servers which handle the state of all the poker hands. In any given hand the player to act gets 10 seconds to act (bet,fold,call,raise,etc.). If there is no response within 10 seconds the server automatically folds for them. To check that 10 seconds has passed an event list of when actions must be received is maintained. It is a priority queue ordered by time and each poker hand currently being played has an entry in the priority queue.
Consider the following scenario since the last action 9.99 seconds pass before the next action arrives at the http server. By the time the action is forwarded to the backend servers extra time passes so now a total of 10.1 seconds have passed. The backend servers will have declared the hand folded, but I would like the action to be processed since technically it arrived at the http server after 9.99 seconds. Now one solution would be to have the backends wait some extra time before declaring a hand folded to see if an action timestamped at 9.99 seconds comes. But that would result in delaying when the next person in the hand gets to act.
The goals I would like are
Handle actions reaching the http server at 9.99 seconds instead of folding their hand.
Aggressively minimize delay resulting from having to do idle waiting to "solve" problem mentioned in bullet point 1.
What are the various solutions? To experts in distributed systems is there known literature on what the trade offs are to various solutions. I would like to know the various solutions deemed acceptable by distributed systems literature. Not just various ad hocs solution.
Maybe on the server side when client request arrives you could take the timestamp?
So you would take "start" and "stop" timestamps, to measure exactly 9.9s?