Can audio being recorded be compressed on the fly?
or are there any libraries to take a .caf file, and convert to a compressed format?
The Core Audio APIs (which is what you'll be using to record audio) allow you to choose format, bitrate, etc. You can choose a low bitrate to keep the filesize down. In most cases with the iPhone, you're recording voice audio, and a low bitrate is fine.
The Audio Queue Services can record and encode audio. These are available on the iPhone. The documentation includes saving to a file of arbitrary format. I don't know if it's a perfect match for transcoding a .caf file but it's definitely worth a look.
The 2.x SDK does not have support for compressing audio.
You'll probably have to compile and link a 3rd party library.
Watch the CPU usage, it obviously won't be hardware
accelerated like the built-in decompression.
Related
Right now my choice is quicktime .mov with H.264 at 30% quality.
I have a small demo video which I want to include, and file-size has the highest priority. I know H.264 is the best choice because it is hardware accelerated. But maybe there is a video format which gives about the same quality with smaller file size?
H.264 will be your format of choice. To save further bits you can use a better encoder, as QuickTime is quite inefficient. The most efficient H.264 encoders are all based on x264.
A nice x264 based open-source option with a GUI and device presets is HandBrake.
a standard mp4 is probably the most compact video out there
I am making a simple application for iPhone, and I want to enter a short audio file on an object click. Which of .caf and .wav would be better?
I am building a simple application in Cocos2d in which balloons produce a pop sound when clicked. What are the memory issues with both sound versions?
If you do not need specific Core Audio Format features, then WAV has more universal support (and it would be my default choice for that reason).
Core Audio Format basically functions as a container for other audio file formats, including WAV. Core Audio Format has many great features, but it's not evident from the description that you need any of these.
In response to a deleted comment, which was moved to the question:
I can't speak for Cocos2d specifically, so I will write about the file formats in general: WAV does not use data compression. CAF may. If it is a short sound file, you probably don't want data compression (because it requires a good amount of processing to convert to LPCM for playback). If you play the pop often, then you will want to hold onto an uncompressed version of the audio data for easy processing. 1 second will require 44100 * 2 bytes at CD quality in memory (per channel).
For a short sound file such as a balloon pop, a 16 bit WAV file sounds ideal. In that sense, the memory difference should not be a deciding factor. If you have a lot of audio files, or long audio files to load into memory, then the situation changes. For now, I don't consider memory to be a problem in your case. Since CAF is a container, its uncompressed representation will be nearly identical (the difference will be a little more header data in the CAF).
A CAF file is a basically Core Audio Format. So it is well suited for the Apple frameworks. The best advantage of CAF over WAV is while recording when you can have files more than 4 GB and also in CAF you don't need to update the WAV header after each packet recording.
Anyway, I assume you don't need these features related to CAF. And as Justin said, I do believe that WAV will be the better option as you can have more support for WAV than the CAF format.
What kind of audio files are you using in your iPhone games/apps?
I have a game with 30MB of sounds in .wav format and I'm thinking of maybe converting to .mp3 to reduce the app size... Is there a major difference in performance? Any other issues?
Keep in mind that certain codecs run in hardware and others in software. Therefore not all compressions will allow for simultaneous playback of more than one sound. For example, if you have a sound playing, a UI sound like a beep may not play if both were trying to use the same codec. For more info, see:
http://developer.apple.com/iphone/library/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/AudioandVideoTechnologies/AudioandVideoTechnologies.html#//apple_ref/doc/uid/TP40007072-CH19-SW6
iPhone Audio Hardware Codecs
iPhone OS applications can use a wide range of audio data formats. Starting in iPhone OS 3.0, most of these formats can use software-based encoding and decoding. You can simultaneously play multiple sounds in all formats, although for performance reasons you should consider which format is best in a given scenario. Hardware decoding generally entails less of a performance impact than software decoding.
The following iPhone OS audio formats can employ hardware decoding for playback:
AAC
ALAC (Apple Lossless)
MP3
The device can play only a single instance of one of these formats at a time through hardware. For example, if you are playing a stereo MP3 sound, a second simultaneous MP3 sound will use software decoding. Similarly, you cannot simultaneously play an AAC and an ALAC sound using hardware. If the iPod application is playing an AAC sound in the background, your application plays AAC, ALAC, and MP3 audio using software decoding.
To play multiple sounds with best performance, or to efficiently play sounds while the iPod is playing in the background, use linear PCM (uncompressed) or IMA4 (compressed) audio.
To learn how to check which hardware and software codecs are available on a device, read the discussion for the kAudioFormatProperty_HardwareCodecCapabilities constant in Audio Format Services Reference.
Both AAC and CAF formats work fine and offer decent file sizes. For certain background looping tracks I found MP3 files getting too big, but YMMV. Experimenting with a decent sound editing app is the only way to find the right balance between size and quality. I've had pretty good luck with Audacity and Amadeus Pro.
Suggest listening to the output with a pair of really good noise-isolating headphones on the device itself. Most people won't be listening to your stuff with these but as you decrease sound quality to shrink file sizes you'll start getting static and hum artifacts. It's just a matter of balancing size vs. quality and what you're willing to live with.
I use a combination of WAV files (for sound effects) and MP3 (for music), which seems to work fine. You can have trouble if you try to play multiple MP3 files at the same time - drop outs, or performance degradation, depending on your AudioSession settings.
If I had to compress my sound effects, I'm not sure which codec has the least decoding overhead. Something like Apple Lossless would likely work well, and would cut the size roughly in half.
I find mp3 fine, but keep in mind that decoding on the iPhone/Touch2G is only about 2.5x realtime speed.
I am using the AVAudioRecorder class available with the 3.0 SDK to record audio in my app.
I am successfully able to record to a caf/aiff file. But I would like to record to an mp3.
I tried changing some values in the settings dictionary but all I get is a blank file.
Is it possible to record to an mp3 using AVAudioRecorder
If not, how can I transfer the caf files from my app to a computer?
Thanks.
See:
http://developer.apple.com/iphone/library/qa/qa2008/qa1615.html
IMA is a 4:1 format, which means it will compress to 25% of the original size. It's good enough for voice, but not ideal for music.
Lossless is great for music, but usually only gets 50%.
µLaw and aLaw are older formats that compress to 50%, but are much worse than Lossless.
I haven't used iLBC.
I'd bet you'll never see an MP3 encoder made available to third-party apps due to licensing issues.
From all I can find on the API documentation, you can't change the encoder. The only encoder settings I could find that you can change are:
AVEncoderAudioQualityKey;
AVEncoderBitRateKey;
AVEncoderBitDepthHintKey;
As for transferring the file, a typical approach would be to email the file.
You'll have to include your own MP3 encoder. And you might not be able to do the encoding in real-time. Sad, but there it is.
I'm having a hard time trying to record something other than linear PCM on the iPhone :-(
The samples I've found (SDK's SpeakHere, Zdziarski's and Sadun's books and the one at trailsinthesand.com) all use linear PCM but I'd like a commonly used compressed format instead (no ima4 or whatever the name is...).
I just cannot figure out how to tweak the sample code to be used with, for example AAC, MP3 or AMR instead. Any suggestions and hints for how to do that are much appreciated!
(Btw, I do not think an MP3-encoder nor AMR-encoder are available due to licensing issues, but AAC does exist, or???)
Edit/Update: I stumbled upon the following text in Apple's "iPhone Application Programming Guide", 2009-01-06, page 137, section: Recording Audio:
"You can record audio in any of the formats listed in “Preferred Audio Formats in iPhone OS” (page 140)", and as preferred audio formats on page 140 are: "For compressed audio when playing one sound at a time, and when you don’t need to play audio simultaneously with the iPod application, use the AAC format packaged in a CAF or m4a file."
Thus, I interpret that as a clear indication that it is indeed, not only possible, but even preferable, to record audio in AAC format wrapped up in a m4a file, which is just what I want. But still, I am not able to achieve that?!
Thanks,
/John
Keep looking at those docs. In "Core Audio Essentials", the section "Core Audio Plug-ins: Audio Units and Codecs" notes that:
iPhone OS contains the recording
codecs listed in Table 2-5. As you can
see, neither MP3 nor AAC recording is
available. This is due to the high CPU
overhead, and consequent battery
drain, of these formats.
Table 2-5 lists several formats, but as the text notes does not include the ones you're looking for. If you want those formats you'll have to bring your own encoder.
It is possible in iPhone 3GS and iPod 2nd generation and above. They have a hardware enconder for AAC.
There is an Apple example project for this that does exactly what you want to do:
iPhoneExtAudioFileConvertTest
A tip for using faac to convert from iPhone-output linear pcm files to aac format:
I found by experimentation that you have to use the -X flag on the command-line version of faac (running on Mac OS X), which "swaps the input bytes." (I guess it changes the endian-ness.)
So, for example, if you audio recorded the file linear.pcm on your iPhone, you could then run:
faac -XP linear.pcm
on your mac to convert that into the aac file linear.aac.
I'm guessing this means you might be able to use faac within your app to do the conversion, if you wanted to, since it's a C library.
Also note that, technically, the files output by iPhone's audio services are usually CAF files, Core Audio Format, which is sort of a wrapper format around a bunch of possible encodings: Apple's docs on CAF. The above command line works fine for me even though linear.pcm starts with the caff header.
Have you tried Lame mp3 encoder? Or faac/faad AAC codecs? You can embed them in ffmpeg for even more audio codecs.
Why don't try TPAACAudioConverter written by the legendary Michael Tyson ?
TPAACAudioConverter is a simple Objective-C class that performs the
conversion of any audio file to an AAC-encoded m4a, asynchronously
with a delegate, or converts any audio provided by a data source class
(which provides for recording straight to AAC).