I would like to fill an array from a stream for around ten seconds.{I wish to do some processing on the data)So far I can:
(a) obtain the microphone stream using mediaRecorder
(b) use analyser and analyser.getFloatTimeDomainData(dataArray) to obtain an array but it is size limited to only a little over half a second of data.I can also successfully output the data after processing back onto a stream and to outDestination.
(c) I have also experimented with obtaining a 'chunks' array from mediaRecorder directly but the problem then is that I can't find any mime type that would give me a simple array of values - ie an uncompressed sample by sample single channel set of value - ie a longer version of 'dataArray' in (b).
I am wondering if I am missing a simple way round this problem?
Solutions I have seen tend to use step (b) and do regular polls then reassemble a longer array - however it seems the timing is a bit tricky ..
I'v also seen suggestions to use audio workouts - I might have to do this but would prefer a simpler solution!
Or again, if someone knows how to drive mediaRecorder to output the chunks array in a simple array format FLOAT32.of one channel.That would do the trick.
Or maybe I'm missing something simpler?
I have code showing those steps that have been successful and will upload if anyone requests.
Related
I have fragmented mp4 which I want to send users through HLS. It's ok if I just send it as is. But I need opportunity to reorder fragments in this video.
For example initial video, which looks like this:
original video format
I want reorganize fragments and get this:
expected video format
I try make it locally, and it's work in VLS player (HLS). For this I modified sequence number for fragments in moof (mfhd). But when I try play it remotely (HLS) it does not work. I think, that some players (js) expect some additional information from each fragment, probably for example time offset. But I can not find which atom (box) contain this information. I spent a lot of time searching and I'm still at the very beginning of the problem.
I tried to modify the fragment sequence number, but it doesn't work.
The "Track Fragment Media Decode Time Box" (tfdt) stores the baseMediaDecodeTime which is the accumulative decode time.
Consider the following...
baseMediaDecodeTime must increase monotonically for each chunk.
This means you must update (replace) the tfdt entry of chunk with expected next tftd entry.
When you naively reorder the chunks, the baseMediaDecodeTime will be invalid.
The "Track Fragment Media Decode Time Box" (tfdt) is located inside each moof header at:
moof --> traf --> tfdt
I'm trying to filter a signal and then analyse the values of the filtered signal using Tone.js / Web-Audio API.
I'm expecting to get values of the filtered signal, but I only get -Infinity, meaning that my connections between the nodes are wrong. I've made a small fiddle demonstrating this, however in my use-case I do not want to send this node to the destination of the context - I only want to analyse it, not hear it.
osc.connect(filter)
filter.connect(gainNode)
gainNode.connect(meter)
console.log(meter.getLevel())
I guess you tested the code in Chrome because there is a problem with Chrome which causes it to not process anything until it is connected to the destination. When using Tone.js that means you need to call .toMaster() at the end of your chain. I updated you fiddle to make it work: https://jsfiddle.net/8f7abzoL/.
In Firefox calling .toMaster() is not necessary therefore the following works in Firefox as well: https://jsfiddle.net/yrjgfdtz/.
After some digging I've found out that I need to have a scriptProcessorNode - which is apparently no longer recommended - so looking into Audio Worklet Nodes
We're making use of the newly added MessageSummaryItems.PreviewText feature. Thank you!!
On issue is: sometimes the PreviewText contains HTML links? From reading through the source I see this in ImapFolderFetch.cs
var body = message.TextBody ?? message.HtmlBody;
So this is saying: use the Plaintext version, if it exists, then use the HTML version?
Therefore if I see links in the preview, I can assume no Plaintext version is available?
Our problem with this is:
If our message only has an HTML version, We could strip the links from the message in our code, but there are only 256 characters of it. In many cases, there will be nothing left to display.
As per your TODO: Using the CONVERT extension would be a better approach but, as far as I can tell its not supported by Gmail?
A fall back would be:
If we could set the preview length for both HTML and Plaintext individually, then we could say, If you only have an HTML version give me 1K of it and i'll strip out the links on the client.
Thoughts?
Very few IMAP servers support the CONVERT extension which is the main reason I didn't implement it.
The PreviewText feature is an attempt at adding a convenience feature to fetch the first 256 bytes of each message body in batched requests in order to minimize latency, but no matter what I do, it's not guaranteed to be useful (since there could be a ton of markup before any real text is included in HTML).
If I were to split text and html messages into 2 different batches so that I could request different sizes for each, then it would be less efficient and might take significantly longer to fetch, so I'm not sure if it's really worth it. The less I'm able to batch at a time, the less useful the feature becomes compared to implementing your own loop over the list of messages and downloading your own specified chunk size. one message at a time.
My suggestion would be to use the PreviewText feature and for the rare messages where the 256 bytes isn't enough, perform a folder.GetStream() on them.
I'm trying to use GDCL MP4 Muxer with my RTSP Source Filter. They work fine together except after stopping the graph, muxer doesn't finilize the file and write the reqiured tables to the end of file via file writer (some parts are written starting from moov but not the time table values). When I try another RTSP source filter (which I don't have its source codes), table values are created with GDCL MP4 Muxer.
But when I try Elecard's MP4 Muxer, it works fine with my RTSP Source Filter. So, there is an incompatibility. I examined GDCL's source codes but couldn't find what it was expecting from me. I already calculate and set timestamp values to samples using SetTime method. But GDCL still doesn't finilaze file. Is it caused by missing information or missing signal when graph stops? What can be the problem, any ideas?
One thing you should be aware of regarding Geraint's MP4 Mux is that it is checking incoming media samples to have both start and stop time. You might be having only .tStart/AM_SAMPLE_TIMEVALID which still makes sense for video, but this would be a problem.
So the samples have to have stop time, or you need to fix this in multiplexer code.
A typical symptom for the problem is that generated files are empty or of zero duration.
We are seeing this when "tying" two video files together.
Example we have Ad video that is segmented and content file which is also segmented.
We create a new file which has both Ad and content segment information together. However we are seeing an issue where either the Ad content is truncated or the content starts having A/V sync issues.
Both ad and content are segmented the same way , 5 sec segmentation. however since Ads are variable length the result file may have left over segment something like:
#EXTM3U
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:5,
fileSequence6.ts
#EXTINF:5,
fileSequence7.ts
#EXTINF:4,
fileSequence8.ts
#EXTINF:5,
fileSequence0.ts
#EXTINF:5,
fileSequence1.ts
#EXTINF:5,
fileSequence2.ts
#EXTINF:3,
fileSequence3.ts
Is this the proper way to play 2 files one after the other without rebuffering?
should generate-variant-plist be used to a play list of 2 files?
When you have a break in the stream to switch to a commercial, ad, or alternate video source then you want to introduce the discontinuity tag before the start of the next segment, for example:
#EXTM3U
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:5,
movie0.ts
#EXTINF:2,
movie1.ts
#EXT-X-DISCONTINUITY
#EXTINF:5,
commercial0.ts
#EXTINF:5,
commercial1.ts
#EXTINF:3,
commercial2.ts
This gets a little more complicated if you encrypt the streams because they use progressive encryption based on the prior segments encryption state and the sequence number which come together to form an "Initialization Vector". If you break the stream you have to reset the initialization vector so that the encryption/decryption can continue uninterrupted. This is an involved process so best to just search on Initialization Vector in Apple's docs.