I have a karaoke app that takes mp3+g files and mixes in the mic (using audio units)... and is able to record the final results to a raw pcm file.
I have potential clients asking me if I can do the same with a karaoke video file. So, can I...
1) playback a video and mix in the microphone audio?
2) take #1 and record it to a video file?
3) possibly even take the video camera and microphone inputs, and video audio... and save this to a file?
Thinking about this can I... take an mp3, camera and microphone inputs and mix all of these to a video file?
Thanks for any feedback!
Brian
Yes, all of these are possible. With the CD+G format, you will be a lot better off converting those to h.264 on the desktop and then loading them as movies with an AAC audio track. Attempting to write a CD+G parser for iOS would be a nightmare since so many CD+G files are badly broken. Also, CD+G files are shocking large, much larger than the MP3 files because CD+G is dumb uncompressed video from the 80's. It is a silly format.
Related
I am making a simple application for iPhone, and I want to enter a short audio file on an object click. Which of .caf and .wav would be better?
I am building a simple application in Cocos2d in which balloons produce a pop sound when clicked. What are the memory issues with both sound versions?
If you do not need specific Core Audio Format features, then WAV has more universal support (and it would be my default choice for that reason).
Core Audio Format basically functions as a container for other audio file formats, including WAV. Core Audio Format has many great features, but it's not evident from the description that you need any of these.
In response to a deleted comment, which was moved to the question:
I can't speak for Cocos2d specifically, so I will write about the file formats in general: WAV does not use data compression. CAF may. If it is a short sound file, you probably don't want data compression (because it requires a good amount of processing to convert to LPCM for playback). If you play the pop often, then you will want to hold onto an uncompressed version of the audio data for easy processing. 1 second will require 44100 * 2 bytes at CD quality in memory (per channel).
For a short sound file such as a balloon pop, a 16 bit WAV file sounds ideal. In that sense, the memory difference should not be a deciding factor. If you have a lot of audio files, or long audio files to load into memory, then the situation changes. For now, I don't consider memory to be a problem in your case. Since CAF is a container, its uncompressed representation will be nearly identical (the difference will be a little more header data in the CAF).
A CAF file is a basically Core Audio Format. So it is well suited for the Apple frameworks. The best advantage of CAF over WAV is while recording when you can have files more than 4 GB and also in CAF you don't need to update the WAV header after each packet recording.
Anyway, I assume you don't need these features related to CAF. And as Justin said, I do believe that WAV will be the better option as you can have more support for WAV than the CAF format.
I have come across some sample codes where set of images are added to make a QTmovie.
I am targeting this for OS X platform without any QT frameworks.
I have ague idea of creating a file with extension and embed it with appropriate metadata and find a way to insert images and audio in required format. So when the file is created it can simply be played.
I am not sure of what format/extension is better.
pointers are much appreciated.
Without QuickTime (or an equivalent multimedia framework), what you describe is quite a lot of work. Ordinarily, you would use a video compression algorithm (such as H.264) to encode your images into video, and an audio compression algorithm (such as AAC) to encode your audio track. Then you would write these streams into a container file, such as an MPEG-4 file, which interleaves the streams for playback, contains metadata and indexes and so on. Then for playback, you parse the file, decode the video and audio data, and schedule them for playback, taking care to keep them in sync.
QuickTime does all this (and more) for you, and it would be an enormous undertaking to write it all yourself. Is there some reason why you are running on OS X but cannot use QuickTime?
Given the question is tagged with iPhone, why can't you just use QTKit?
If you had to do it from scratch, you could adopt a very simple solution whereby you store your image sequence as a set of JPEG files (but then you would require libjpeg; use raw RGB or PPM if you must), the audio track as a raw WAV data, and then have another file (a text file you define) that stored timing information, so you would simply stream out the audio, and have the frame numbers of the images stored with their corresponding timecode/sample offset. That is a very simple solution that could be made to work without too much effort.
If you give us some more idea of what you are trying to achieve, we could offer some more specific suggestions.
If you want to write a program to do this, you could use Xuggler in Java to do it. It will allow you to save your final video in a format playable by almost any media player.
Start out by gaining an understanding of how video files (e.g. MP4, Quicktime) actually represent audio and video with this Overly Simplistic Guide to Internet Video.
Then, play around with the MediaTool tutorials. You can write programs that make raw images into video files (see this sample code). Finally, to write a program that makes audio and video that are in sync, see this tutorial; it generates a set of images, and makes some audio noise that is timed to change when a ball hits the edge of a box.
Hope that helps.
Art
I have implement game application in which i want to play mid sound file in the background.How it possible?
The short answer is there is no easy way. When this question came up before the best recommendation was to try porting TiMidity++. This appears to be difficult.
A more iPhoney solution would be to convert your midi file into an audio file using something such as
Easy-Pro Midi to Audio Converter 1.3
and then playing the the audio file normally in your app. This may have the downside of enlarging your app size.
I was using AVAudioPlayer to play multiple audio clips back to back but there was always a small silence between tracks and then i came to know of Finch, a library which uses OpenAL to play audio. with this the silence problem seems to be solved theoretically but then i found that it doesn't play m4a or any other compressed formats.
Now i am looking for an uncompressed audio format which would have relatively less file size (though uncompressed means that all of them should have almost same size) and a method to convert, i am also googling on afconvert in a mean while.
CAF files work great for this. I've built an application that loops audio files, and I was impressed with the relatively small file size.
Check out this question for more info on converting to CAF.
What kind of audio files are you using in your iPhone games/apps?
I have a game with 30MB of sounds in .wav format and I'm thinking of maybe converting to .mp3 to reduce the app size... Is there a major difference in performance? Any other issues?
Keep in mind that certain codecs run in hardware and others in software. Therefore not all compressions will allow for simultaneous playback of more than one sound. For example, if you have a sound playing, a UI sound like a beep may not play if both were trying to use the same codec. For more info, see:
http://developer.apple.com/iphone/library/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/AudioandVideoTechnologies/AudioandVideoTechnologies.html#//apple_ref/doc/uid/TP40007072-CH19-SW6
iPhone Audio Hardware Codecs
iPhone OS applications can use a wide range of audio data formats. Starting in iPhone OS 3.0, most of these formats can use software-based encoding and decoding. You can simultaneously play multiple sounds in all formats, although for performance reasons you should consider which format is best in a given scenario. Hardware decoding generally entails less of a performance impact than software decoding.
The following iPhone OS audio formats can employ hardware decoding for playback:
AAC
ALAC (Apple Lossless)
MP3
The device can play only a single instance of one of these formats at a time through hardware. For example, if you are playing a stereo MP3 sound, a second simultaneous MP3 sound will use software decoding. Similarly, you cannot simultaneously play an AAC and an ALAC sound using hardware. If the iPod application is playing an AAC sound in the background, your application plays AAC, ALAC, and MP3 audio using software decoding.
To play multiple sounds with best performance, or to efficiently play sounds while the iPod is playing in the background, use linear PCM (uncompressed) or IMA4 (compressed) audio.
To learn how to check which hardware and software codecs are available on a device, read the discussion for the kAudioFormatProperty_HardwareCodecCapabilities constant in Audio Format Services Reference.
Both AAC and CAF formats work fine and offer decent file sizes. For certain background looping tracks I found MP3 files getting too big, but YMMV. Experimenting with a decent sound editing app is the only way to find the right balance between size and quality. I've had pretty good luck with Audacity and Amadeus Pro.
Suggest listening to the output with a pair of really good noise-isolating headphones on the device itself. Most people won't be listening to your stuff with these but as you decrease sound quality to shrink file sizes you'll start getting static and hum artifacts. It's just a matter of balancing size vs. quality and what you're willing to live with.
I use a combination of WAV files (for sound effects) and MP3 (for music), which seems to work fine. You can have trouble if you try to play multiple MP3 files at the same time - drop outs, or performance degradation, depending on your AudioSession settings.
If I had to compress my sound effects, I'm not sure which codec has the least decoding overhead. Something like Apple Lossless would likely work well, and would cut the size roughly in half.
I find mp3 fine, but keep in mind that decoding on the iPhone/Touch2G is only about 2.5x realtime speed.