I am writing an iPhone application to apply filters to audio input and output the result in real time.
I am new to audio processing but using audiounit, the correct approach?
I found out how to output data using audiounit but couldn’t figure out how to capture input audio.
Is there a sample application showing how to connect input and output using audiounit?
Check out the aurioTouch sample app from apple. Audio unit is a very complex framework to use. I suggest you look at the AVFoundation framework if it suits your need. I hope it helps you.
Related
I am writing an app that generates music. I am using OpenAL to: modify gain; modify pitch; mix audio; and play the resulting audio. I now need to record the audio as it is being played. I understand that OpenAL does not let you record the output audio. The other options I have found is to use audio units. However because I need to mix/pitch/gain the audio and record it, it seems I need to write all the audio processing so I can have access to the output buffer. Is this correct? Or is there a different iOS API I can use to do this. If not then is there a 3rd party solution already that lets me record the output (paid solutions are fine)?
You are correct.
Audio Units are the only iOS public API that allows an app to both process and then record audio.
Trying to record the OpenAL output may well be a violation of Apple's rules against using non-public APIs.
The alternative may be to completely rewrite the portions of OpenAL you need (there may be open source for some portions) running on top of the RemoteIO Audio Unit.
The best way to go is likely to be Core Audio, since it will give you as much flexibility as you need. Take a look into the Extended Audio File Services reference pages.
Using and extended audio file you should be able to set up a file format and audio stream buffer to send the final mixed output to, and then use the ExtAudioFileWrite() function to write the samples to the file.
I need to play and record simultaneously using Core Audio. I really do not want to use AVFoundation API (AVAudioPlayer + AVAudioRecorder) to do this as I am making a music app and cannot have any latency issues.
I've looked at the following source code from Apple:
aurioTouch
MixerHost
I've already looked into the following posts:
iOS: Sample code for simultaneous record and playback
Record and play audio Simultaneously
I am still not clear on how I can do playback and record the same thing simultaneously using Core Audio. Any pointers towards how I can achieve this will be greatly appreciable. Any pointers to any sample source code will also be of great help.
The RemoteIO Audio Unit can be used for simultaneous record and play. There are plenty of examples of recording using RemoteIO (aurioTouch) and playing using RemoteIO. Just enable both unit input and unit output, and handle both buffer callbacks. See an example here
On iOS, is it possible to get the user's audio stream in a decompressed format? For example, the MP3 is returned as a WAV that can be used for audio analysis? I'm relatively new to the iOS platform, and I remember seeing that this wasn't possible in older iOS versions. I read that iOS 4 brought in some advanced APIs but I'm not sure where I can find documentations/samples for these.
If you don't mind using API for iOS 4.1 and above, you could try using the AVAssetReader class and friends. In this similar question you have a full example on how to extract video frames. I would expect the same to work for audio, and the nice thing is that the reader deals with all the details of decompression. You can even do composition with AVComposition to merge several streams.
These classes are part of the AVFramework, which allows not only reading but also creating your own content.
Apple has an OpenAL example at http://developer.apple.com/library/mac/#samplecode/OpenALExample/Introduction/Intro.html where Scene.m should interest you.
The Apple documentation has this picture where the Core Audio framework clearly shows that it gives you MP3 out. It also states that you can access audio units in a more radical way if you so need.
The same Core Audio document gives also some information about using MIDI if it may help you.
Edit:
You're in luck today.
In this example an audio file is loaded and fed into an AudioUnit graph. You could fairly easily write an AudioUnit of your own to put into this graph and which analyzes the PCM stream as you see fit. You can even do it in the callback function, although that's probably not a good idea because callbacks are encouraged to be as simple as possible.
In this link it says it does:
http://developer.apple.com/library/ios/#documentation/AudioVideo/Conceptual/MultimediaPG/UsingAudio/UsingAudio.html
However I have searched for an example that works on the web and can only see people complaining they can't get it to work. I have a working AudioQueue example for PCM but the very moment I switch this to AAC initialization fails. SpeakHere example also only uses PCM.
Has anyone ever managed to make this work or has a link to a code snippet that works?
Basically iPhone only record PCM. To get an AAC encoded file the original PCM stream should be stored into a AudioFile and then this AudioFile should be converted to AAC.
There's no way to record and convert on the fly on iPhone unless you include a third-party library that will do this for you.
I want to code up a audio fx program that would process the samples coming in through the mic input and play them through the audio output in near realtime. But I cannot seem to figure out how to get accesses to the individual audio samples as they come in. Can. Anybody point me in the right direction and possibly some sample code? Thanks.
The audio-processing framework in iOS is called Core Audio. See the Core Audio Overview. For sample code see the iOS Reference Library sample code. You can filter for audio using the searchbox
You probably want to be looking at the Audio Queue services (most/all of the function names begin with "AudioQueue").