What I have
My aim is to parse some media file using ffmpeg and provide video and audio playback. Which I do successfully using the OpenGL for video and AudioQueue for audio.
What I need to do
I need to change AudioQueue to Audio Unit service, because it does provide several nasty features for Audio manipulations.
Basically I'm confused on integration of Audio Units into ffmpeg run loop.
So would like to have some references/samples from you guys where Audio Unit is intergrated with ffmpeg media playback loop i.e. media packet extraction and its pushing into some buffer which Audio Unit can play.
Yes, i already used AudioUnit for playing a decoded audio using ffmpeg.
I took Novocaine project from github ( https://github.com/alexbw/novocaine ) and did a some changes (mostly threw code for input and OSX).
See KxAudioManager class from kxmovie project: https://github.com/kolyvan/kxmovie
If you plan to code for iOS, then note about need resampling sound to 44100Hz.
A many movies have audio streams with 48000 Hz sample rate.
And the best result I got: resample via swr_convert function from libswresample lib.
Related
I am trying to write an app for a school project that will play a video stream of the lab. They use Foscam cameras, and from what I understand they only stream audio in FFMPEG and VLC (a wrapper of FFMPEG and others, correct?). I can play the video stream in a UIWebView just fine, but I can't figure out how to get audio to work. I have read a bunch of post on here about the different audio stream options, but none support this format. I have also looked into VLCKit, but that seems to only work in OS X, not iOS.
Is there a Framework, or Kit that can stream IMA WAV ADPCM audio from a VLC/FFMPEG file, or does someone know of a better way to get audio from a Foscam?
EDIT - I know more about the stream and I have tried a few things. I tried AVPlayer, but discovered Audio FIle Stream and Audio Queue Services but can't find a good tutorial.
I also found THIS site, and tried the project with my stream and it did not work.
I'm trying to get samples from AudioQueue to show spectrum of music (like in iTunes) on iPhone.
Ive read a lot of posts but almost all asks about get samples when Recording, not playing :(
I'm using AudioQueue Services for streaming audio. Please help to understanding next points:
1/ Where can I get access to samples (PCM, non mp3 (I'm using mp3 stream)
2/ Should I collect samples in my own buffer to apply fft ?
3/ Is it possible get frequencies without fft transformations ?
4/ How can I synchronize my fft shift in buffer with current playing samples ?
thanks,
update:
AudioQueueProcessingTapNew
For iOS6+, this works fine for me. But what about iOS5 ?
For playing audio, the idea is to get at the samples before you feed them to the Audio Queue callback. You may need to convert any compressed audio file format into raw PCM samples beforehand. This can be done using one of the AVFoundation converter or file reader services.
You can then copy frames of data from the same source used to feed the Audio Queue callback buffers, and apply your FFT or other DSP for visualization to them.
You can use either FFTs or a bank of band-pass filters to get frequency info, but the FFT is very efficient at this.
Synchronization needs to done by trial-and-error, as Apple does not specify exact audio and view graphic display latencies, which may differ between iOS devices and OS versions anyway. But short Audio Queue buffers or using the RemoteIO Audio Unit may give you better control of the audio latency, and OpenGL ES will give you better control of the graphic latency.
I'm a DirectSound programmer new to iOS. I want to implement the ability to play streaming multichannel audio, sometimes looping back to a specified point when the stream is finished playing (think of a song that has a little intro diddy that is played once, then the song loops indefinitely, skipping that intro).
With DirectSound and libvorbis, at least, I'd feed a chunk of the OGG data into the libvorbis decoder, it'd spit out some PCM, and I'd fill the buffer and queue it up to play right after the current sound buffer is finished, swapping between two buffers.
Probably looking at using some kind of hardware-supported format on iOS, like AAC. What programming APIs should I be using that will allow me to do multichannel and loop points? Any input is appreciated, thanks!
The iOS AVAssetReader class can be used to read compressed audio file data into a PCM buffer. Either the Audio Queue API (simpler) or the RemoteIO Audio Unit (lower latency) can be used to play buffers of PCM data.
I am trying to record the sound from iPhone speaker. I am able to do that, but I am unable to avoid mic input in the recorded output. Tried with sample code available in different websites with no luck.
The sample which I used does the recording with audio units. I need to know if there is any property for audio unit to set the mic input volume to zero. Above that I came to from other posts that Audio Queue services can do the thing for me. Can any one redirect me with sample code for the audio queue services implementation. I need to know whether there is a way of writing the data to an separate audio file before sending it as input to speaker.
Thanks in advance
There is no public iOS API or property for recording generic audio sent to the iPhone speaker. Only mic input can be recorded.
But if you are playing audio in your app using only uncompressed samples with Audio Queues or the RemoteIO Audio Unit, you can just copy those samples to a file before you write them to the audio callback buffers. Those saved samples can be used to construct a recording.
I have sucessfully made Audio Capturer and Audio Player using audio unit.
Now my requirement is to make a conference call where i have one audio Capturer but multiple audio Players ...When testing i can only hear sound from one player. Can i make Multiple player with single audio unit ? Plz help
Linear PCM and IMA4 (IMA/ADPCM) You can play multiple linear PCM or IMA4 sounds simultaneously in iOS without incurring CPU resource problems. The same is true for the iLBC speech-quality format, and for the ยต-law and a-law compressed formats. When using compressed formats, check the sound quality to ensure it meets your needs. Ref
Using Linear PCM it should be able to play simultaneously.
Another solution I can think of is use a Mixer Audio Unit and send all your players to the Mixer and play the output.