The definition of an Ogg/Opus stream requires two headers at the beginning of the stream. These headers are required by decoders, so it is impossible for one to pick up a long-running stream in the middle. If the Opus headers could be repeated periodically, it would be possible for a receiver to be attached to an Opus stream anywhere, and start decoding when the headers appeared, but I haven't found anything in the RFCs or other docs that would allow this.
It would be possible to insert headers by ending and restarting the stream, thus breaking the stream into a number of short pieces, but I don't know if existing decoders would output the result as a continuous stream of decoded audio or not.
Is there is a way to structure a long-running Ogg/Opus stream so existing players (e.g., vlc) could play it correctly regardless of where they happen to pick it up?
Related
We have a peculiar scenario we want to test.
We'll be consuming an HTTP stream which would stay open during certain time-frame. The stream consists of plain-text lines (CSV) and it's streamed using the chunked transfer encoding.
When we connect we expect to get all the data from, possibly, a file on the server side, and once that bulk is being served the connection stays alive, as it's possible that there would be more data transferred over the same connection.
Is it possible for Wiremock to serve everything from a file and keep the connection alive (doesn't send an empty chunk to signal the end of stream)?
The short answer is no.
While WireMock will keep connections alive by default per the HTTP 1.1 spec, it will always terminate the response once everything has been sent, either via the empty chunk or by setting Content-Length.
What you're trying to do (if I understand correctly) is stream out multiple payloads within the context of a single response, which WireMock doesn't have a means for doing.
A possible solution might be for you to concatenate all your response parts into a single file, although I suspect you've discounted that option for reasons not stated.
Another possibility would be to supply your own FileSource implementation to WireMock and thus provide your own InputStreamSource which would give you more control over how the underlying file(s) are streamed out in the response.
I'm using Mido for python, working on parsing midi files into <start_time, duration, program, pitch> tuples and met some problems.
Some files that I parse has multiple note_on, resulting in notes at the same pitch and same program being opened more than once.
Some files contains multiple note_off resulting in trying to close notes that is no longer on due to being closed before (assuming only one note at the same program and same pitch can be on).
Some tracks does not have a program_change in the beginning of the track (or even worse, not even having one in the whole track).
Some files has more than one track containing set_tempo messages.
What should I do in each of these cases to ensure I get the correct interpretation?
In general, to get a correct MIDI message stream, you have to merge all tracks in a type 1 file. What matters for a synthesizer are not tracks, but channels.
The MIDI specification says:
ASSIGNMENT OF NOTE ON/OFF COMMANDS
If an instrument receives two or more Note On messages with the same key number and MIDI channel, it must make a determination of how to handle the additional Note Ons. It is up to the receiver as to whether the same voice or another voice will be sounded, or if the messages will be ignored. The transmitter, however, must send a corresponding Note Off message for every Note On sent. If the transmitter were to send only one Note Off message, and if the receiver in fact assigned the two Note On messages to different voices, then one note would linger. Since there is no harm or negative side effect in sending redundant Note Off messages this is the recommended practice.
The General MIDI System Level 1 Developer Guidelines say that in response to a “GM System On” message, a device should set Program Change to 0. So you can assume this to be the initial value for channels that have notes without a preceding Program Change.
The Standard MIDI Files specification says that
tempo information should always be stored in the first MTrk chunk.
But "should" is not "must".
I need to copy many small file streams into one file stream. Each file stream carries its own designated position, i.e. FS1 goes firstly, then FS2 and so on. But while multi-threading the program, the thread which finishes the processing firstly adds it firstly which causes errors.
Is there any way that we can define its position so that no matter what sequence we add them, they will get in the right place??
I tried this by creating many headers before hand so that the file streams will replace those headers, but searching for those headers just slows down the program.
This question is in continuation to my last one, as First processedFS jumps (Copies) first, so we need to define location where it will be copied.
Please refer to this question:
Sequential MT
You can't have multiple threads write to the same file stream at the same time without wrapping it with a synchronization lock, and you would also need to re-seek the stream back and forth depending on which thread needs to write to it at any given moment, so it writes at the correct offset within the file. That is a lot of overhead.
You can, however, have multiple threads use different file streams to write to the same file at the same time, provided the sharing rights between the streams are compatible to allow concurrent writing and preserve data. Pre-size the file to the desired length, then divide up sections of that length amongst the threads as needed. Give each thread its own stream to the target file, first seeking to the appropriate start offset of its assigned section. Then each thread can write to its respective stream normally, without having to synch with the other threads or re-seek its stream. Just make sure each thread does not exceed the bounds of its assigned section within the file so it does not overwrite another thread's data.
I know very similar questions have been asked before. But I don't think the solutions I found on google/stackoverflow are suitable for me.
I started to write some web services with Scala/Spray, and it seems the best way to send large files without consuming large amouns of memory is using the stream marshalling. This way Spray will send http chunks. Two questions:
Is it possible to send the file without using HTTP chunks and without reading the entire file into memory?
AFAIK akka.io only process one write at a time, meaning it can buffer one write until it has been passed on to the O/S kernel in full. Would it be possible to tell Spray, for each HTTP response, the length of the content? Thereafter Spray would ask for new data (through akka messages) untill the entire content length is completed. Eg, I indicate my content length is 100 bytes. Spray sends a message asking for data to my actor, I provide 50 bytes. Once this data is passed on to the O/S, spray sends another message asking for new data. I provide the remaining 50 bytes... the response is completed then.
Is it possible to send the file without using HTTP chunks [on the wire]
Yes, you need to enable chunkless streaming. See http://spray.io/documentation/1.2.4/spray-routing/advanced-topics/response-streaming/
Chunkless streaming works regardless whether you use the Stream marshaller or provide the response as MessageChunks yourself. See the below example.
without reading the entire file into memory
Yes, that should work if you supply data as a Stream[Array[Byte]] or Stream[ByteString].
[...] Thereafter Spray would ask for new data [...]
That's actually almost like it already works: If you manually provide the chunks you can request a custom Ack message that will be delivered back to you when the spray-can layer is able to process the next part. See this example for how to stream from a spray route.
I indicate my content length is 100 bytes
A note upfront: In HTTP you don't strictly need to specify a content-length for responses because a response body can be delimited by closing the connection which is what spray does if chunkless streaming is enable. However, if you don't want to close the connection (because you would lose this persistent connection) you can now specify an explicit Content-Length header in your ChunkedResponseStart message (see #802) which will prevent the closing of the connection.
I used a ts analyzer for a .ts file i have with mpeg-2 codec and i found out that it splits in 7311 packets.
I m trying to find this through matlab by using fopen to open the ts file in binary and fread to read the file but all i get is a column with a huge collection of numbers(way above the number of packets). Does anyone know how can i determine which of these data are the packets? Or if someone knows another way to find the packets would help me a lot.
Thank you in advance
From some quick googling, the MPEG-2 transport stream ('ts') format consists of packets 188-bytes in length, each having a 4-byte header followed by a 184-byte payload. Essentially, you can count the number of packets by counting the number of headers you find - but beware that, if you are only interested in counting the number of, e.g., video packets in the stream, then you will need some deeper analysis of the headers, because the stream may contain any number of interleaved "elementary streams" (which can be video, audio, or arbitrary data). Each elementary packet type in the stream is denoted by a unique "PID" which is contained in the header.
Aside from the above, you will also have to handle synchronisation - each header begins with the "synchronisation byte", which has a value 0x47 (or 01000111 in binary). According to this resource, decoders begin by looking for this synchronisation byte; once they find one, they may have found a packet header. To make sure, they try to find three consecutive synchronisation bytes (188 bytes apart in the stream); if three are found, synchronisation can occur and the packet boundaries may from then on be assumed at 188-byte intervals. Note, however, that the first byte of each assumed header should be checked to see if it is a synchronisation byte - if it is not, then this is called "sync loss" and the syncrhonisation process must start again.
Once you have some code to syncrhonise to a stream, it should be fairly easy to extract the PIDs from the header of each packet and count the number of packets associated with each unique PID you find. You should probably also check the first bit after the synchronisation byte as, if set to 1, this indicates a transport error, and the packet's payload is invalid. Detailed information on the format of packet headers can be found here.