I'm making a game for iOS platform and i want to use openAL for playing audio effects in the game (except the background music). I want to have for example 20-30 sounds with duration of 1-3 seconds each. Because i want these sounds to be played with no delay i have to load from file, decompress and store in the memory. But decompressed audio (as i understand) uses a lot of memory. So am I on the right way? Or there is another way ? Thanks
Loading sounds does incur a delay, and can cause the rest of the app to jitter. For best performance you definitely want to preload sound effects.
Memory use is a concern, but as long as you're not loading too much audio data into memory at a time, you'll be OK.
44KHz mono audio data will occupy 88,000 bytes per second when uncompressed. Stereo is double that, but usually for sound effects you don't want stereo anyway. So if you had 30 sounds loaded, each of 3 second duration, you'd have 90 seconds of sound using 7.5MB of memory. You can of course halve that memory usage by using a 22050 Hz source before compressing it to AAC (which preserves the source sample rate).
What I do is maintain a cache of audio buffers that I can flush when the app starts to use too much memory like so: https://github.com/kstenerud/ObjectAL-for-iPhone/blob/master/ObjectAL/ObjectAL/OALSimpleAudio.m#L441
For the lowest latency sound effects, you will want to play uncompressed audio already in memory. 2 or 3 MBytes is not a lot of memory to allocate for low latency sound effects.
Related
I am using cocos2d's CCRenderTexture to record video of my game. But if recording video in retina display resolution will cost lot of CPU and memory, so I want to use low resolution for video record but keep retina-resolution for normal game play. is it possible?
I've tried "[[CCDirector sharedDirector] enableRetinaDisplay:NO];" during record video, but it seems not work. the generated output totally wrong.
This is not feasible.
You'd have to render each frame twice, once on the screen, then onto the render texture. A serious drop in framerate is inevitable even if you lower the resolution of the render texture somehow.
The reason is simply that you'll also have to write each render texture as an image to flash memory. This is extremely slow. You'll also end up with a huge amount of data. If each (PNG/JPG) image file ends up being a reasonably small 50 KB then one second of recorded data at 60 fps will consume 3 Megabytes of flash memory. One minute would be around 180 Megabytes.
To record a demo of your game, most games follow the simple principle of recording the user input, and then playing back the user input as if the user had issued these commands. This requires careful planning, no breaking changes when updating the app (or invalidating old demos), and no use of non-deterministic randomizers (ie seeded with time).
If you need to record a demo for making a trailer video, there's plenty of screengrabbing solutions around. Some even specialize in grabbing iPhone video, either from the device (usually requires a source code/library component) or from the Simulator.
You should check out Kamcord SDK for recording game play. Check at http://kamcord.com/
Kamcord has a built-in gameplay video and audio recording technology for iOS. It allows you, the game developer, to capture gameplay videos with an API. Your users can then replay and share these gameplay videos via YouTube, Facebook, Twitter, and email.
I have a Query regarding playing audio: Can I rewind the audio 30 secs and Play with 2X speed? Is that possible like in Podcasts in iPhone?
Thanks
If you want to speed up sound without making the pitch high and squeaky, then you may need to license a commercial DSP time-pitch stretch library such as Dirac, et.al. (There may be some open source code to do this in Audacity, but I am unaware of a working iOS port of such).
I'm struggling with an AudioQueue audio player I implemented. I initially thought it was truncating the 1st 1/2 of audio that it played but upon loading larger files I notice gaps every other 1/2-1 second. I've run it in debug and I've confirmed that I'm loading the queue correctly with audio. (There are no big zero regions loaded in the queue.) It plays without issue (no gaps) on the simulator but on device I get gaps as if its missing every other chunk of audio. In my app I decompress then pull audio from a memory NSMutableData object. I feed this data into the audio queue. I have a corresponding implementation in the same app that plays wave audio and this example works without issue on long and short audio clips. I'm comparing the wave implementation to the other which does the decompression. the only difference between the two is how I discover the audio meta data and where I get the audio samples for enqueuing. In the wave implementation I use AudioFileGetProperty and AudioFileReadPackets to get this data. In the other case I derive the data before hand using cached ivars loaded during callbacks from my decompressor. The meta data matches for both compressed and wave implementations. I've run the code in instruments and I don't see anything taking more than 1ms in my audio packet delivery/enqueuing logic during playback. I'm completely lost. Please speak up if you have any idea how to solve the situation.
I finally resolved this issue. I found that if I skipped the 1st 44 bytes (the exact size of a wave header) of audio then it plays correctly on the device. It pays correctly on the sim regardless of wether I skip 44 or not. Strange and I'm not sure why but that's the way it works.
Does it load the entire file at a time? Or does it act lazy and load a chunk at a time? I'm primarily interested in knowing how much memory my audio uses.
From what I understand of it, it buffers the file and you can have even more control over this process by using Audio Queues. Most implementations of OpenAL will load the entire file all at once which can be pretty intensive.
You can pre-buffer using prepareToPlay but honestly, I've never had any noticeable lag using wav's, caff's or mp4's. Calling the play method has to pre-buffer the audio anyways.
In all of my uses of AVAudioPlayer, it normally only causes a temporary jump in memory allocation. As long as you release the player after it is finished, memory isn't a problem. I've played as many as 15 sounds at once and never had issue.
More info on audio: http://developer.apple.com/iphone/library/documentation/iphone/conceptual/iphoneosprogrammingguide/AudioandVideoTechnologies/AudioandVideoTechnologies.html
I'm working on a project which is using Audio Toolbox for recording and playback of PCM data, and I'm having trouble with playback. In the simulator, I can record and play audio just fine, using a custom class to handle storing and sourcing PCM bytes for the recording and playback buffers as needed. On device (iPhone (3.0.1) and iPod 2G (3.1.2)) recording works fine, the audio files produced are correct, but in-app playback stutters, like it's only playing part of each playback buffer. My buffers are one second long, and I've got 3 buffers, which are preloaded before playback starts; stuttering occurs during those first 3 seconds as well, which I think rules out a latency problem.
I've written Audio Toolbox code before that worked, and I'm not doing anything strange here except that I'm using my own class to source PCM data instead of AudioFileReadBytes()
I know the data that comes out of my source is good, because it plays right in the sim, and it writes to disk as a correct audio file
I've played around with sample rates a bit; I'm normally using 11025Hz sampling to cut down on file size (it's all voice, so it sounds fine). at 44100Hz, but with the same size of buffers, I get the same stuttering problem, but the audio segments come a lot faster, about 4 times faster. That's why I think it's only playing part of each buffer.
The only reason I can conceive that it would only play part of each buffer is a latency problem... like the audio toolbox code is running out of full buffers while I'm still filling an empty one. But that would cause it to play the preloaded buffers correctly, and then start stuttering, and that doesn't happen, it stutters the whole way through
I've tried humongous buffers, like 10MB buffers, and I just get silence and a single stutter of audio at the end of playback. I've also tried preloading more buffers than normal, like 10 seconds worth of audio, and it behaves the same.
The audio session is being set with AVAudioSession, not the Audio Toolbox calls, and it's being set to the Playback category for playback
I have no idea how to try and attack this problem, it makes no sense to me that it works fine on the simulator but not the device.
Code for the playing callback and the set up for the audio queue services: http://pastebin.com/mfaa546c
It turns out that the use of NSData's GetBytes:length: was causing the problem. The buffer filled with that method was playing incorrectly. However, doing a memcpy from that buffer to another buffer would prevent the problem.