I am currently developing an app that will contain large amounts of audio, around 60-120 minutes. Most voice audio files. My question is really what is the best way to go about storing them. For example, one single large file, separate audio files, download-as-needed cache files.
Any suggestions on file format?
These are the audio formats decoded by iPhone hardware that should take the least power to play.
Other iPhone OS audio formats employ a hardware codec for playback.
These formats are:
AAC
ALAC (Apple Lossless)
MP3
Whether to have the audio distributed with the app or separately would depend on the use. If you could reasonably expect the user to go through the material sequentially, you may want to allow the user to download part by part or stream the audio to let them conserve space on their device, while if the audio is more random access, you'd probably want it all on the device.
Several apps, including Apple's own, appear to use the open source speex codec for compressed voice-quality audio, even though this seems not to be supported by the hardware or any public API.
As Joachim suggested you can choose from AAC/ALAC/MP3 audio formats. What I'd propose now is to also consider the issue from user experience point of view:
Convert all your audio to chosen format with quality options that
satisfy you and your potential users.
Next, calculate the size of all your files and ask yourself a questions:
"are X megabytes too much to bundle for my kind of app?" and
"will that big/small app bundle encourage users to download my app?".
Optionally play a bit with quality options to shrink files (iterate).
In the next step, decide (based on you app characteristics) whether to bundle all files. For example a game is expected to have all files in place and can be big (users accept that). If your app has e.g. podcasts only, then select the best one and bundle it - once user is hooked he can download the rest (let user trigger that), so files are stored on device. Also provide user the info how much data they are about to download and warn them if file is reasonably big and they're not on Wi-Fi; or introduce the option to download only on Wi-Fi.
I hope that sounds reasonable.
For music, the following approach would be much different.
Since it's just voice, you can reduce the sample rate significantly in the majority of cases. Try [8kHz…20kHz].
In case they are multichannel - Mono should be fine for voice.
Once that's been done, I'd recommend AAC for size and quality balance.
Do some listening tests on your devices. Tweak settings if needed. Then batch process/convert them all. That can reduce your sizes by ten or more if the sources are 16/44.1.
Unless they files are very small (e.g. seconds each) or you have to open and read many of them quickly, I wouldn't bother with the huge file. A few MB is a good size for many cases.
Related
According to this What Techniques Are Best To Live Stream iPhone Video Camera Data To a Computer? is possible to get compressed data from iphone camera, but as I've been reading in the AVFoundation reference you only get uncompressed data.
So the questions are:
1) How to get compressed frames and audio from iPhone's camera?
2) Encoding uncompressed frames with ffmpeg's API is fast enough for real-time streaming?
Any help will be really appreciated.
Thanks.
You most likely already know....
1) How to get compressed frames and audio from iPhone's camera?
You can not do this. The AVFoundation API has prevented this from every angle. I even tried named pipes, and some other sneaky unix foo. No such luck. You have no choice but to write it to file. In your linked post a user suggest setting up the callback to deliver encoded frames. As far as I am aware this is not possible for H.264 streams. The capture delegate will deliver images encoded in a specific pixel format. It is the Movie Writers and AVAssetWriter that do the encoding.
2) Encoding uncompressed frames with ffmpeg's API is fast enough for real-time streaming?
Yes it is. However, you will have to use libx264 which gets you into GPL territory. That is not exactly compatible with the app store.
I would suggest using AVFoundation and AVAssetWriter for efficiency reasons.
I agree with Steve. I'd add that on trying with Apple's API, you're going to have to do some seriously nasty hacking. AVAssetWriter by default spends a second before spilling its buffer to file. I haven't found a way to change that with settings. The way around that seems to be to force small file writes and file close with the use of multiple AVAssetWriters. But then that introduces lots of overhead. It's not pretty.
Definitely file a new feature request with Apple (if you're an iOS developer). The more of us that do, the more likely they'll add some sort of writer that can write to a buffer and/or to a stream.
One addition I'd make to what Steve said on the x264 GPL issue is that I think you can get a commercial license for that which is better than GPL, but of course costs you money. But that means you could still use it and get pretty OK results, and not have to open up your own app source. Not as good as an augmented Apple API using their hardware codecs, but not bad.
I know this is not a specific programming question but I hope someone can give me a suggestion. My applications (iPhone and Blackberry applications) use a lot of audio files. I need a solution for my applications in order to save some spaces.
Is it right that .aac is the most suitable audio format for iPhone? Is it the smallest one? It it also suitable for Blackberry?
Is there any way to make the audio files smaller without losing a lot of quality of the sounds? How about the bitrate, sampling freq and channels? Are they really matter?
AAC is a good format for the iPhone. The iOS is optimized to play AAC.
Yes, things like bitrate, sampling frequency and number of channels are all factors in the audio file's size.
What you should do is take your audio and convert it to different formats with different settings and then just play them on a real device to see if the quality is acceptable.
Sorry, there is no simple answer. Experiment.
Depends on what type of audio you're encoding. For speech, AMR is supported by all major smartphones, and will generally give the smallest file sizes. Quality degredation is noticeable enough that it's not suitable for music, but it's optimized for voice recording (the voice notes app on the BlackBerry uses it as its file format) so it'll give you very nice results with spoken audio.
What I need to do is be able to mix 4 channels of audio (not from a live source, just prerecorded audio files in the app bundle), and change their volumes individually, in real time, preferably with MP3s. What's the best/correct road for me to take, regarding all the various sound APIs for the iPhone?
Thanks!
Storm Sim does this with AVAudioPlayer, which is certainly the simplest methdod. You can call prepareToPlay on each of the player objects then kick them off with play later so there won't be any delay. I also use a blank 1-second audio player on eternal loop to keep the deviceTime counting down, so you can use playAfter to give a specific deviceTime in the future to make all the samples play in-sync or offset relative to each other (deviceTime only ticks if there is some sort of audio playing). The AVAudioPlayerDelegate has interrupted/resumed events and finishedPlaying so you can get notification of what is happening.
However there is only one hardware MP3/AAC decoder, so the other three will use up CPU (and thus battery) doing the decoding. If you want to maximize battery life, use CAF files in IMA4#44100. It is about 1/4 the size of the raw WAV files so it isn't as good as MP3 but the performance is much better, especially if you are using a lot of small audio tracks. If you are using voice you can get away with much less fidelity and smash the files even more. afconvert in terminal can help you getting your source files in the CAF format (you should use CAF files no matter what the encoding).
I have come across some sample codes where set of images are added to make a QTmovie.
I am targeting this for OS X platform without any QT frameworks.
I have ague idea of creating a file with extension and embed it with appropriate metadata and find a way to insert images and audio in required format. So when the file is created it can simply be played.
I am not sure of what format/extension is better.
pointers are much appreciated.
Without QuickTime (or an equivalent multimedia framework), what you describe is quite a lot of work. Ordinarily, you would use a video compression algorithm (such as H.264) to encode your images into video, and an audio compression algorithm (such as AAC) to encode your audio track. Then you would write these streams into a container file, such as an MPEG-4 file, which interleaves the streams for playback, contains metadata and indexes and so on. Then for playback, you parse the file, decode the video and audio data, and schedule them for playback, taking care to keep them in sync.
QuickTime does all this (and more) for you, and it would be an enormous undertaking to write it all yourself. Is there some reason why you are running on OS X but cannot use QuickTime?
Given the question is tagged with iPhone, why can't you just use QTKit?
If you had to do it from scratch, you could adopt a very simple solution whereby you store your image sequence as a set of JPEG files (but then you would require libjpeg; use raw RGB or PPM if you must), the audio track as a raw WAV data, and then have another file (a text file you define) that stored timing information, so you would simply stream out the audio, and have the frame numbers of the images stored with their corresponding timecode/sample offset. That is a very simple solution that could be made to work without too much effort.
If you give us some more idea of what you are trying to achieve, we could offer some more specific suggestions.
If you want to write a program to do this, you could use Xuggler in Java to do it. It will allow you to save your final video in a format playable by almost any media player.
Start out by gaining an understanding of how video files (e.g. MP4, Quicktime) actually represent audio and video with this Overly Simplistic Guide to Internet Video.
Then, play around with the MediaTool tutorials. You can write programs that make raw images into video files (see this sample code). Finally, to write a program that makes audio and video that are in sync, see this tutorial; it generates a set of images, and makes some audio noise that is timed to change when a ball hits the edge of a box.
Hope that helps.
Art
I am using the AVAudioRecorder class available with the 3.0 SDK to record audio in my app.
I am successfully able to record to a caf/aiff file. But I would like to record to an mp3.
I tried changing some values in the settings dictionary but all I get is a blank file.
Is it possible to record to an mp3 using AVAudioRecorder
If not, how can I transfer the caf files from my app to a computer?
Thanks.
See:
http://developer.apple.com/iphone/library/qa/qa2008/qa1615.html
IMA is a 4:1 format, which means it will compress to 25% of the original size. It's good enough for voice, but not ideal for music.
Lossless is great for music, but usually only gets 50%.
µLaw and aLaw are older formats that compress to 50%, but are much worse than Lossless.
I haven't used iLBC.
I'd bet you'll never see an MP3 encoder made available to third-party apps due to licensing issues.
From all I can find on the API documentation, you can't change the encoder. The only encoder settings I could find that you can change are:
AVEncoderAudioQualityKey;
AVEncoderBitRateKey;
AVEncoderBitDepthHintKey;
As for transferring the file, a typical approach would be to email the file.
You'll have to include your own MP3 encoder. And you might not be able to do the encoding in real-time. Sad, but there it is.