I can receive from the camera a packet that contains information about the frame. So, I know its size. The rest of the packet is an array of bytes (uint8).
I would like to know how to manage this packet of bytes and how to decode it in order to get a frame. I know from the header that the frame can be I-frame or P-frame and that its size is 640x360. The encoding can be set to H264 or MP4. I'm working with MatLab script, which is an M-file. If MatLab cannot manage this array of bytes directly, I would like to know how a byte stream works, to be able to re-assamble the frame myself.
Related
I want to playback sensor data from different sensors in matlab synchronously. For most data I use updating plots. However, I have difficulties to find an efficient way to playback a sequence of jpeg data as video on screen (720p or 1080p).
One way i tried is to use the matlab function to convert the jpeg data to raw matlab images (https://nl.mathworks.com/help/robotics/ref/readimage.html) and then display it with the imshow function. This way is very slow and therefore not suitable.
Another way was to write the jpeg data to a file (fwrite) and using the matlab system command to execute ffplay:
ffplay -loglevel panic -framerate 30 -i output.jpg
This works for showing single .jpeg's, but is becoming tricky when I want to play a sequence of jpeg data as a movie. If I write all jpeg data sequentially to a file and run ffplay afterwards, I can playback the movie correctly. But what I want to playback on the fly (i.e. stream) and not wait until all data is written to the file before playback.
Is there a suitable way to share the jpeg data from matlab directly to ffplay in order to play a sequency of jpeg data on-the-fly? Any other suggestions to tackle this problem are also welcome.
I have an instrument transmitting data in packets of 40 characters each time.
Each packet includes a packet of interest (32 characters in the middle of it) that is bounded by additional four predefined characters before and after it used in to locate it.
The data is transmitted to the PC to a comport buffer. I want to read the data from it using matlab:
Is there a way to scan the content of a buffer continuously in the background without blocking the command line?
Is it possible to read data from the buffer as soon as a packet exists in it by finding the first and last characters that define each packet?
i want help in OPUS DTX
1.how OPUS DTX is working.
2.how opus encoder and decoder works for dtx
3.Is there any bit representation for DTX in OPUS
From rfc When DTX is enabled, only one frame is encoded
every 400 milliseconds it didnt works for me with opus_demo binary.
please help with OPUS DTX.
When using Opus over a network, using a protocol such as RTP where the packets are timestamped, DTX may be enabled if you want to reduce the packets sent during periods where there is no voice activity. A packet would still be sent about every 400 ms, updating background noise. Using the packet timestamps the receiver can determine the duration of any gaps and fill them in with the background noise to keep it sounding natural.
Enable DTX in the encoder using opus_encoder_ctl(enc, OPUS_SET_DTX(1));, or with the -dtx option on opus_demo. Then, simply do not send any packets produced by the encoder with a length of 2 bytes or less. (These "DTX packets" are just zero-length frames, with a normal 1- or 2-byte packet header (TOC), and do not contain any audio data.) Packets with a length larger than 2 bytes should be sent as usual.
The receiver should use normal packet loss concealment to handle missing packets, in the same manner as it would handle packet loss. In particular it can call opus_decode() with data = NULL, len = 0, and frame_size equal to the size of the missing frame(s), and the decoder will generate audio data to conceal the missing frame(s).
I want to generate a sound wave programmatically and play it with AVAudioPlayer. I have the code to encode my waveform as linear PCM, 44100Hz, mono, 8 bits per sample.
I am not clear on what kind of envelope I need to wrap around this buffer so that AVAudioPlayer recognizes it as PCM.
PCM is just a digital representation of an analog audio signal. Unfortunately, it doesn't encapsulate any of the metadata about the audio - channels, bit depth, or sample rate - all necessary to properly read said PCM data. I would assume AVAudioPlayer would accept this PCM data wrapped in an NSData object as long as you were able to set those variables manually in the AVAudioPlayer object. Unfortunately, those variables are read only, so even though the documentation says AVAudioPlayer can handle anything that Core Audio can handle, it has no way to handle raw LPCM data.
As stated by zoul, I would imagine that the easiest way to go about this is throwing in a WAV header, since the header informs AVPlayer of the above necessary variables. It's 44 bytes, is easily mocked up, and is defined nicely - I used the same definition given above to implement wav header encoding and decoding. Also, it's just prepended to your unmodified LPCM data.
Maybe adding a WAV header would help?
I posted a Swift 5 example (as a GitHub Gist) of converting a buffer of audio float samples into an in-memory WAV file to use with AVAudioPlayer initWithData, here: https://gist.github.com/hotpaw2/4eb1ca16c138178113816e78b14dde8b
I'm making an Iphone game, we need to use a compressed format for sound, and we want to be able to loop SEAMLESSLY back to a specific sample in the audio file (so there is an intro, then it loops back to an offset)
currently THE ONLY export process I have found that will allow seamless looping (reports the right priming and padding frame numbers, no clicking when looping ect) is using apple's afconvert to a aac format in a caf file.
but when we try and encode to lower bitrates, it automatically re samples the sound! we do NOT want to have the sound re sampled, every other encoder I have encountered has an option to set the output sample rate, but I can't find it for this one.
on another note, if anyone has had any luck with seamless looping of a compressed file format using audio queues, let me know.
currently I'm working off the information found at:
http://developer.apple.com/mac/library/qa/qa2009/qa1636.html
note that this DID work PERFECTLY when I left the bitrate for the encode at default (~128kbs) but when I set it to 32kbps - with the -b option - it resampled, and looping clicks now.
It needs to be at least 48kbps. 32kbps will downsample to a lower sample rate.
I think you are confusing sample rate (typical values: 32kHz, 44.1kHz, 48kHz) and bit rate (typical values: 128kbps, 160kbps, 192kbps).
For a bit rate, 32kbps is extremely low. Sound will have bad quality at this bit rate. You probably intended to set the sample rate to 32kHz instead, which is also not outright typical, but makes more sense.
When compressing to AAC and uncompressing back to WAV, you will not get the same audio file back, because in AAC, the audio data is represented in a completely different format than in raw wave. E.g. you can have shifts by few microseconds, which are necessary to convert to the compressed format. You can not completely get around this with any highly compressed format.
The clicking sound originates from the sudden change between two samples which are played in direct succession. This is likely taking place because the offset to which you jump back in your loop does not end up to be at exactly the same position in the AAC file as it was in the WAV file (as explained above, there can shifts by microseconds).
You will not get around these slight changes when compressing. Instead, you have to compensate for them after compression by adjusting the offset. That means you have to open the compressed sound file in an audio editor, e.g. Audacity, and manually find another offset close to the original one, which is suitable for looping.
How to find an offset which is suitable for looping?
Zoom in to the waveform's end. Look at how the waveform looks there. Then zoom in to the waveform at the original offset and search in its neighbourhood for an offset at which the waveform connects seamlessly to the end of the waveform.
For an example how this shoud look like, open the uncompressed audio file in the audio editor and examine the end of the waveform and the offset there.