Does Exoplayer download a chunk completely before processing (decrypting, decoding) it. Is there a way to override this and start decoding / playback before the chunk is completely downloaded.The content is an MPEG-DASH content with a 6 second chunk size.
I am on the latest version of Exoplayer. I am trying to improve the Video Start Time and hence this query. Also, will smaller chunk sizes impact the Video start time ?
I think you mean a dash segment when you say chunk - the terminology is important because DASH segments can contain subsegments, and each of these may be decodable, but it is also confusing as the term chunks and segments are both used in the ExoPlayer code.
Its useful when discussing this area to remember that the video download is actually a series of requests and responses, rather than a constant stream of media.
To start decoding earlier you typically have to request smaller 'pieces' (trying to avoid tripping over terminology...) of the video.
To be decodable, and video piece usually needs to start with a frame which does not reference any previous frames - an IDR frame or Stream Access Point (SAP).
Looking at ExoPlayer itself, you can set the number of segments you can download per chunk (Exoplayer terminology for the bit of the video you download) - take a look at the 'maxSegmentsPerLoad' attribute in the 'DefaultDashChunkSource': https://github.com/google/ExoPlayer/blob/bd54394391b0527893f382c9d641b8a55ffca765/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashChunkSource.java
However, I think this is the opposite of what you are looking for - you would like to request a smaller piece of video - e.g. subsegments rather than the whole segments.
For that you most likely want to look at the new low latency mechanisms introduced for DASH and for HLS - ExoPlayer has added support for these and there is a public design document which provides a very good explanation of the background and the approach here (link correct at the time of writing - original ExoPlauer git issue also for reference - https://github.com/google/ExoPlayer/issues/4904):
https://docs.google.com/document/d/1z9qwuP7ff9sf3DZboXnhEF9hzW3Ng5rfJVqlGn8N38k/edit#
The diagrams in this document explain it well, but the short answer to your question is that yes, this approach does allow smaller 'pieces' of video be downloaded and played back and it does indeed help with video start time.
Related
I am making an app that uses many large texts that are predefined. At the moment, I am storing them in a RTF file and loading them from there to separate code from content. If the rtf file will have around 1500-2000 lines and it is loaded and split several times when the app launches, will the app need very much time to load and should I define the content in the source-code?
Thanks for your help!
I definitely would not be inclined to put 2000+ lines of text in the source code itself. A separate file (or database or whatever) makes a lot more sense.
Regarding whether the app will require much time to load it, you should simply benchmark it (e.g. put let start = CFAbsoluteTimeGetCurrent() before the load and let elapsed = CFAbsoluteTimeGetCurrent() - start after, and see how long it takes.
By the way, if iOS, make sure you do this on an actual device, not the simulator to get accurate times. But usually loading a small file like this will be inconsequential regarding time required (e.g. it’s likely still going to be smaller than many images).
I just tested loading a fairly heavily marked up rtf (every word a different color), which was 250kb. Using the NSAttributedString initializer that takes a file URL, while it was an order of magnitude slower than a plain text file (same text, no markup), it still only took 70ms on my iPhone. But clearly, you should benchmark your own file.
I have 2 questions
1)How do I write raw data to a file sinker. I am trying to mux.
2)How do I make sure the sinked data is not written to a file but to a memory buffer
So in Detail:
I am trying to use windows MPEG-4 File Sink to write some Intel SDK Encoded avc or hevc to memory and send it to websocket.
what is the right approach?
Can I just feed raw hevc or avc as (byte*, length) to MPEG-4 File Sink?
Or Do I need to wrap the Intel Encoder into a Custom Windows Media Foundation Encoder(well I can just use GUID to get the Intel Encoder anyway) from Windows Media Frame work. Correct me If I am wrong please.
So I have 2 problems, How do I write my raw data(avc||hevc) to MP4 Sinker(Encoded by a 3rd Party Encoder)
Do I need to implement a custom Sinker , And how custom is it. Can I inherit part of the MPEG4 Sinker(After all I do not want to re implement a full container for Mp4)
Or Modify MPEG4 Sinker behavior so that it does not write it to a file but writes to a Memory
I know I feel like I re iterated myself A few times. Sorry about that.
1) If you wrap the encoded bitstream in an IMFSample you can just call IMFStreamSink::ProcessSample. To wrap it in the IMFSample, create a memory buffer IMFMediaBuffer with MFCreateMemoryBuffer , then create an IMFSample with MFCreateSample and add the buffer to it with IMFSample::AddBuffer. And then pass it to the stream sink. Also, if you can constrain the output bitstream length you can actually use the underlying memofy from IMFMediaBuffer by using IMFMediaBuffer::Lock to obtain the pointer to the underlying memory and passing that to the Intel SDK.
2) When creating the MPEG-4 sink via MFCreateMPEG4MediaSink you pass in an IMFByteStream instance. You can make your own class which implements this interface and writes the data directly to memory or wherever you need. If you do not want to do a full implementation there are also MFCreateMFByteStreamOnStream and MFCreateMFByteStreamOnStreamEx which can wrap an IStream instance into a IMFByteStream but I have never used those and I am not aware of the underlying memory semantics. You can create a memory backed IStream with SHCreateMemStream and CreateStreamOnHGlobal.
I have used Intel SDK quite long ago but if I remember it had a MFT compatible encoder, but I always used the plain C++ one, and thus I am not sure how they differ in terms of configuration etc. But if the MFT one works, then you can setup a proper pipeline without processing the bitstream samples yourself as stated in (1) and just handle (2).
Also, performance wise, since as far as I remember Intel SDK did work on Direct3D surfaces as well, you could look into MFCreateDXSurfaceBuffer to used Direct3D surfaces instead of memory buffers for wrapping the data.
I want to decode a MP3 file. I manage to find the 32 bits in the header (sync word, ID, Layer, Bitrate, etc). The problem is I have no idea on how to find the starting (the position) of main_data_begin (side information). I am using MATLAB in this case.
I know it may be a simple question, but I really need your help. Please.
Thank you.
MPEG1/2 Layer III uses main_data_begin as a kind of pseudo-VBR over the granule headers & data. The simplest way to do it is to implement a circular buffer that receives all the physical frame data after the side info and throws-away the unused bytes at the beginning of the buffer (as indicated by main_data_begin) before starting frame decode.
Your best bet is to read an existing decoder's source. The spec is also really good for this, but main_data_begin is mis-documented in publicly-available versions (as best as I can find).
I'm wondering why Vorbis needs any container at all? I know you can stick Vorbis in an Ogg container or a Matroska container, but if I'm not going to bundle it with any video or any other multimedia why can't the Vorbis data stand alone in its own file?
Has anyone had any experience doing this? I googled before searching SO and I only found a single mention in the oggvorbis mailing list with no details.
It is completely possible. You do not need to know before hand the length of any Vorbis packet (whether they are headers or audio) to be able to decode them. Without the Ogg wrapper (or an alternative wrapper) you will miss out on a few things, but they might not be important for your application:
The page checksums - not too important if you are reading from a disk or other rather reliable source
The page granule/last sample positions - useful for improving seeking performance, and for specifying
However, you can pretty trivially make a pure Vorbis bytestream from a ogg file (given there is only one Vorbis stream in it) by:
Skipping 26 bytes
n = read 1 byte
countOfBytesToRead = sum of next n bytes
Read countOfBytesToRead bytes into your Vorbis bytestream
Repeat 1-4 until Ogg file is exhausted
The following function calls are deprecated in OpenAL 1.1, what is a proper replacement?? THe only answer i found in google was "write your own function!!" ;-)
alutLoadWAVFile
alutUnloadWAV
There are 8 file loading functions in ALUT (not including the three deprecated functions alutLoadWAVFile, alutLoadWAVMemory, and alutUnloadWAV).
The prefix of the function determines where the data is going; four of them start alutCreateBuffer (create a new buffer and put the sound data into it), and the other four start alutLoadMemory (allocate a new memory region and put the sound data into it).
The suffix of the function determines where the data comes from. Your options are FromFile (from a file!), FromFileImage (from a memory region), HelloWorld (fixed internal data of someone saying "Hello, world!"), and Waveform (generate a waveform).
I believe the correct replacement for alutLoadWAVFile would therefore be alutCreateBufferFromFile.
However, I would not use this blindly - it's suitable for short sound clips, but for e.g. a music track you probably want to load it in chunks and queue up multiple buffers, to ease the memory load.
These functions are all covered in the alut documentation, by the way.
"write your own" is pretty much the correct answer.
You can usually get away with using the deprecated functions since most implementations still include the WAV file handling functions, with one notable exception being iOS, for which you'd need to use audio file services.
I'd suggest making a standard prototype for "load wav file" and then depending on the OS, use a different loading routine. You can just stub it with a call to alutLoadWAVFile for systems known to still support it.