When reading multiple bytes from an I2C-based RTC, it seems that it is possible that while reading each byte, one of the values may increment.
For instance, if the time is:
2014-12-31 23:59:59
as you're reading this value, the time may roll-over to
2015-01-01 00:00:00
so you may actually read:
2015-01-01 23:59:59
(depending on which values you read first).
So, is it the rtc driver's responsibility to ensure a reliable read?
Reading the datasheet for the DS1337, page 9 states:
When reading or writing the time and date registers, secondary (user)
buffers are used to prevent errors when the internal registers update.
When reading the time and date registers, the user buffers are
synchronized to the internal registers on any start or stop and when
the register pointer rolls over to zero.
Therefore, if reading (or writing) occurs with a single I2C operation (without wrapping around), the RTC device guarantees that everything is synchronized.
[I haven't examined the datasheets for any other devices, but I assume they all work similarly.]
Related
I am investigating the clock sources that common posix objects wait on. I am worried that our code is affected by time changes by user or services like ntp.
It is mostly CLOCK_REALTIME or "just the system clock".
E.g:
The timeout shall be based on the CLOCK_REALTIME clock. The resolution
of the timeout shall be the resolution of the clock on which it is
based. The timespec data type is defined in the <time.h> header.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutex_timedlock.html
This means it is definitely affected by time changes.
I cannot find this information on sockets. There is receive timeout (sockopt SO_RCVTIMEO), linger timeout (sockopt SO_LINGER) and especially select().
I am particularly interested in select() as it can also be called on other file descriptors like mqueues.
I've recently upgraded my kafka streams from 2.0.1 to 2.5.0. As a result I'm seeing a lot of warnings like the following:
org.apache.kafka.streams.kstream.internals.KStreamWindowAggregate$KStreamWindowAggregateProcessor Skipping record for expired window. key=[325233] topic=[MY_TOPIC] partition=[20] offset=[661798621] timestamp=[1600041596350] window=[1600041570000,1600041600000) expiration=[1600059629913] streamTime=[1600145999913]
There seem to be new logic in the KStreamWindowAggregate class that checks if a window has closed. If it has been closed the messages are skipped. Compared to 2.0.1 these messages where still processed.
Question
Is there a way to get the same behavior like before? I'm seeing lots of gaps in my data with this upgrade and not sure how to solve this, as previously these gaps where not seen.
The aggregate function that I'm using already deals with windowing and as a result with expired windows. How does this new logic relate to this expiring windows?
Update
While further exploring I indeed see it to be related to the graceperiod in ms. It seems that in my custom timestampextractor (that has the logic to use the timestamp from the payload instead of the normal timestamp), I'm able to see that the incoming timestamp for the expired window warnings indeed is bigger than the 24 hours compared to the event time from the payload.
I assume this is caused by consumer lags of over 24 hours.
The timestamp extractor extract method has a partition time which according to the docs:
partitionTime the highest extracted valid timestamp of the current record's partition˙ (could be -1 if unknown)
so is this the create time of the record on the topic? And is there a way to influence this in a way that my records are no longer skipped?
Compared to 2.0.1 these messages where still processed.
That is a little bit surprising (even if I would need to double check the code), at least for the default config. By default, store retention time is set to 24h, and thus in 2.0.1 older messages than 24h should also not be processed as the corresponding state got purged already. If you did change the store retention time (via Materialized#withRetention) to a larger value, you would also need to increase the window grace period via TimeWindows#grace() method accordingly.
The aggregate function that I'm using already deals with windowing and as a result with expired windows. How does this new logic relate to this expiring windows?
Not sure what you mean by this or how you actually do this? The old and new logic are similar with regard to how a long a window is stored (retention time config). The new part is the grace period that you can increase to the same value as retention time if you wish).
About "partition time": it is computed base on whatever TimestampExtractor returns. For your case, it's the max of whatever you extracted from the message payload.
I want to read and encode MIDI .mid files at the byte (and bit level). When reading through a .mid file, how do I recognize that a specific byte is the first byte of a delta time value?
For example, below is Figure 2.12 from Mandal's Multimedia Signals and Systems a diagram of a track chunk of a .mid file:
How do I know that the 01, 01, 78, 00, 00, and 00 are delta time bytes, given that the events they are attached to are of varying byte lengths? (For example, the instrument change appears to be two bytes beyond the delta time byte, while the first Note On event appears to contain 3 bytes beyond the delta time byte). Is it based on interpreting what follows the delta time byte? If so the fact that the second Note On event is throwing me: why does it appear to have only two bytes following the delta time byte?
It does not appear in Mandel's example, but I would appreciate an answer that clarified this for multi-byte delta times also.
And of course, I appreciate input on how to improve my question.
Finally, I am not asking for libraries that will automate reading .mid files. Those are good (yay!) but I want to understand the how to read the file format down to the byte level.
You indeed have to decoce a MIDI message before you know where the next delta time begins.
The actual Standard MIDI Files specification says:
<MTrk event> = <delta-time> <event>
<delta-time> is stored as a variable-length quantity. It represents the amount of time before
the following event. If the first event in a track occurs at the very beginning of a track, or if
two events occur simultaneously, a delta-time of zero is used. Delta-times are always present.
[…]
<event> = <MIDI event> | <sysex event> | <meta-event>
<MIDI event> is any MIDI channel message. Running status is used: status bytes of MIDI
channel messages may be omitted if the preceding event is a MIDI channel message with the
same status. The first event in each MTrk chunk must specify status. Delta-time is not
considered an event itself: it is an integral part of the syntax for an MTrk event. Notice that
running status occurs across delta-times.
I came across this term, "power-fail interrupt" in wikipedia here
Since power-failure occurs instantly, what is the use of this interrupt? By the time the computer realizes it has encountered a power-failure, it'll be switched off! What's the point of it? Also this interrupt is given the highest priorityHave I misunderstood the term "power-fail"?
After a power failure, the computer can remain on life support using reserve power (such as a battery).
The article you quote talks specifically about the VAX architecture. The way the power-fail interrupt works there is as follows:
Once a power-failure interrupt has been posted, the processor has
approximately 4 milliseconds before power is shut down.
Thus the interrupt handler has four milliseconds to do its thing.
When listening for MidiEvents in NAudio from a MidiDevice, we get the long "AbsoluteTime" property on each event. But what unit is this time in and from what starting point is it measured?
In a MIDI file, each event has a delta in "ticks" since the last event. To make MIDI files easier to work with, NAudio keeps a running total, storing the value in AbsoluteTime. The meaning of this depends on delta ticks per quarter note (which is a property on the MidiFile class), and the tempo (MIDI files ought to include at least one TempoEvent).
When listening for MIDI events from a device, the AbsoluteTime of the MIDI Event created will be 0. However, you can use the TimeStamp property of the MidiInMessageEventArgs which I believe is in milliseconds since MidiInStart was called.