OpenAL vs AVAudioPlayer vs other techniques for playing sounds - sprite-kit

I know that OpenAL is fast library but it doesn't support any compressed audio format and it's not so easy to use...
AVAudioPlayer is not so fast, but supports wide range file formats, as well as compressed formats like mp3.
Also there is an SKAction class which can play a sound, as well as SystemSoundID...
I have few questions:
What would be a preferred way/player/technique to play :
sound effects(multiple at the time)?
sound effects which can be sometimes repeated after small time delay
background music that loops
Also, is it smart move to use uncompressed audio for sound effects? I suppose this is ok, because those files have small size anyway?

I'm personally using ObjectAL. It's good because it utilizes OpenAL and AVAudioPlayer but abstracts a lot of the complicated parts away from you. My game has background music, tons of sounds playing simultaneously, and loopable sounds that increase in volume, pitch etc based on a sprites speed. ObjectAL can do all of that.
ObjectAL can be used for playing simple sounds and music loops using it's OALSimpleAudio class. Or you can get deeper into it and do more complex things.
I've created a simple wrapper around ObjectAL specifically for my game so it's further abstracted away from me.
From what I've read, uncompressed audio is better. You just need to make sure you preload the sounds so that your game isnt trying to pull the file each time it's playing a sound.

This very simple class has serviced multiple projects for me flawlessly with lots of sounds running at the same time. I find it a lot simpler than fussing with OpenAL. It has everything you asked for, pre-setup sounds, multiple concurrent plays, after delay, background loops.
Doesnt matter if you use compressed files or not because you set it up first.
SKAudio.h
#import <Foundation/Foundation.h>
#import AVFoundation;
#interface SKAudio : NSObject
+(AVAudioPlayer*)setupRepeatingSound:(NSString*)file volume:(float)volume;
+(AVAudioPlayer*)setupSound:(NSString*)file volume:(float)volume;
+(void)playSound:(AVAudioPlayer*)player;
+(void)playSound:(AVAudioPlayer*)player afterDelay:(float)delaySeconds;
+(void)pauseSound:(AVAudioPlayer*)player;
#end
SKAudio.m
#import "SKAudio.h"
#implementation SKAudio
#pragma mark -
#pragma mark setup sound
// get a repeating sound
+(AVAudioPlayer*)setupRepeatingSound:(NSString*)file volume:(float)volume {
AVAudioPlayer *s = [self setupSound:file volume:volume];
s.numberOfLoops = -1;
return s;
}
// setup a sound
+(AVAudioPlayer*)setupSound:(NSString*)file volume:(float)volume{
NSError *error;
NSURL *url = [[NSBundle mainBundle] URLForResource:file withExtension:nil];
AVAudioPlayer *s = [[AVAudioPlayer alloc] initWithContentsOfURL:url error:&error];
s.numberOfLoops = 0;
s.volume = volume;
[s prepareToPlay];
return s;
}
#pragma mark sound controls
// play a sound now through GCD
+(void)playSound:(AVAudioPlayer*)player {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
[player play];
});
}
// play a sound later through GCD
+(void)playSound:(AVAudioPlayer*)player afterDelay:(float)delaySeconds {
dispatch_time_t popTime = dispatch_time(DISPATCH_TIME_NOW, delaySeconds * NSEC_PER_SEC);
dispatch_after(popTime, dispatch_get_main_queue(), ^(void){
[player play];
});
}
// pause a currently running sound (mostly for background music)
+(void)pauseSound:(AVAudioPlayer*)player {
[player pause];
}
#end
To use in your game:
Set up a class variable and pre-load it with your sound:
static AVAudioPlayer *whooshHit;
static AVAudioPlayer *bgMusic;
+(void)preloadShared {
// cache all the sounds in this class
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
whooshHit = [SKAudio setupSound:#"whoosh-hit-chime-1.mp3" volume:1.0];
// setup background sound with a lower volume
bgMusic = [SKAudio setupRepeatingSound:#"background.mp3" volume:0.3];
});
}
...
// whoosh with delay
[SKAudio playSound:whooshHit afterDelay:1.0];
...
// whoosh and shrink SKAction
SKAction *whooshAndShrink = [SKAction group:#[
[SKAction runBlock:^{ [SKAudio playStarSound:whooshHit afterDelay:1.0]; }],
[SKAction scaleTo:0 duration:1.0]]];
...
[SKAudio playSound:bgMusic];
...
[SKAudio pauseSound:bgMusic];

Here's a port of #patrick's solution to Swift 3.
import AVFoundation
// MARK -
// MARK setup sound
// get a repeating sound
func setupRepeatingSound(file: String, volume: Float) -> AVAudioPlayer? {
let sound: AVAudioPlayer? = setupSound(file: file, volume: volume)
sound?.numberOfLoops = -1
return sound
}
// setup a sound
func setupSound(file: String, volume: Float) -> AVAudioPlayer? {
var sound: AVAudioPlayer?
if let path = Bundle.main.path(forResource: file, ofType:nil) {
let url = NSURL(fileURLWithPath: path)
do {
sound = try AVAudioPlayer(contentsOf: url as URL)
} catch {
// couldn't load file :(
}
}
sound?.numberOfLoops = 0
sound?.volume = volume
sound?.prepareToPlay()
return sound
}
// MARK sound controls
// play a sound now through GCD
func playSound(_ sound: AVAudioPlayer?) {
if sound != nil {
DispatchQueue.global(qos: .default).async {
sound!.play()
}
}
}
// play a sound later through GCD
func playSound(_ sound: AVAudioPlayer?, afterDelay: Float) {
if sound != nil {
DispatchQueue.main.asyncAfter(deadline: .now() + Double(afterDelay)) {
sound!.play()
}
}
}
// pause a currently running sound (mostly for background music)
func pauseSound(_ sound: AVAudioPlayer?) {
sound?.pause()
}

Related

Is it possible to play video using Avplayer in Background?

I am using Avplayer to show video clips and when i go back (app in background) video stop. How can i keep playing the video?
I have search about background task & background thread ,IOS only support music in background (Not video)
http://developer.apple.com/library/ios/#documentation/iphone/conceptual/iphoneosprogrammingguide/ManagingYourApplicationsFlow/ManagingYourApplicationsFlow.html
here is some discussion about play video in background
1) https://discussions.apple.com/thread/2799090?start=0&tstart=0
2) http://www.cocoawithlove.com/2011/04/background-audio-through-ios-movie.html
But there are many apps in AppStore, that play video in Background like
Swift Player : https://itunes.apple.com/us/app/swift-player-speed-up-video/id545216639?mt=8&ign-mpt=uo%3D2
SpeedUpTV : https://itunes.apple.com/ua/app/speeduptv/id386986953?mt=8
This method supports all the possibilities:
Screen locked by the user;
List item
Home button pressed;
As long as you have an instance of AVPlayer running iOS prevents auto lock of the device.
First you need to configure the application to support audio background from the Info.plist file adding in the UIBackgroundModes array the audio element.
Then put in your AppDelegate.m into
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions:
these methods
[[AVAudioSession sharedInstance] setDelegate: self];
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:nil];
and #import < AVFoundation/AVFoundation.h >
Then in your view controller that controls AVPlayer
-(void)viewDidAppear:(BOOL)animated
{
[super viewDidAppear:animated];
[[UIApplication sharedApplication] beginReceivingRemoteControlEvents];
[self becomeFirstResponder];
}
and
- (void)viewWillDisappear:(BOOL)animated
{
[mPlayer pause];
[super viewWillDisappear:animated];
[[UIApplication sharedApplication] endReceivingRemoteControlEvents];
[self resignFirstResponder];
}
then respond to the
- (void)remoteControlReceivedWithEvent:(UIEvent *)event {
switch (event.subtype) {
case UIEventSubtypeRemoteControlTogglePlayPause:
if([mPlayer rate] == 0){
[mPlayer play];
} else {
[mPlayer pause];
}
break;
case UIEventSubtypeRemoteControlPlay:
[mPlayer play];
break;
case UIEventSubtypeRemoteControlPause:
[mPlayer pause];
break;
default:
break;
}
}
Another trick is needed to resume the reproduction if the user presses the home button (in which case the reproduction is suspended with a fade out).
When you control the reproduction of the video (I have play methods) set
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(applicationDidEnterBackground:) name:UIApplicationDidEnterBackgroundNotification object:nil];
and the corresponding method to be invoked that will launch a timer and resume the reproduction.
- (void)applicationDidEnterBackground:(NSNotification *)notification
{
[mPlayer performSelector:#selector(play) withObject:nil afterDelay:0.01];
}
Its works for me to play video in Backgorund.
Thanks to all.
If you try to change the background mode:
Sorry, App store wont approve it.MPMoviePlayerViewController playback video after going to background for youtube
In my research, someone would take the sound track out to play in te background when it goes into background as the video would be pause and get the playbacktime for resume playing when go into foreground
It is not possible to play background music/video using Avplayer. But it is possible using
MPMoviePlayerViewController. I have done this in one of my app using this player & this app
is run successfully to appstore.
Try with this snippet, I've already integrated this with my app & it's being useful for me..hope this will work for you!!
Follow the steps given below:
Add UIBackgroundModes in the APPNAME-Info.plist, with the selection
App plays audio
Then add the AudioToolBox framework to the folder frameworks.
In the APPNAMEAppDelegate.h add:
-- #import < AVFoundation/AVFoundation.h>
-- #import < AudioToolbox/AudioToolbox.h>
In the APPNAMEAppDelegate.m add the following:
// Set AudioSession
NSError *sessionError = nil;
[[AVAudioSession sharedInstance] setDelegate:self];
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:&sessionError];
/* Pick any one of them */
// 1. Overriding the output audio route
//UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
//AudioSessionSetProperty(kAudioSessionProperty_OverrideAudioRoute,
sizeof(audioRouteOverride), &audioRouteOverride);
========================================================================
// 2. Changing the default output audio route
UInt32 doChangeDefaultRoute = 1;
AudioSessionSetProperty(kAudioSessionProperty_OverrideCategoryDefaultToSpeaker, sizeof(doChangeDefaultRoute), &doChangeDefaultRoute);
into the
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
but before the two lines:
[self.window addSubview:viewController.view];
[self.window makeKeyAndVisible];
Enjoy Programming!!
Swift version for the accepted answer.
In the delegate:
AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, error: nil)
AVAudioSession.sharedInstance().setActive(true, error: nil)
In the view controller that controls AVPlayer
override func viewDidAppear(animated: Bool) {
UIApplication.sharedApplication().beginReceivingRemoteControlEvents()
self.becomeFirstResponder()
}
override func viewWillDisappear(animated: Bool) {
mPlayer.pause()
UIApplication.sharedApplication().endReceivingRemoteControlEvents()
self.resignFirstResponder()
}
Don't forget to "import AVFoundation"
In addition to fattomhk's response, here's what you can do to achieve video forwarding to the time it should be after your application comes in foreground:
Get currentPlaybackTime of playing video when go to background and store it in lastPlayBackTime
Store the time when application goes to background (in userDefault probably)
Again get the time when application comes in foreground
Calculate the duration between background and foreground time
Set current playback time of video to lastPlayBackTime + duration
If you are Playing video using WebView you can handle using javascript to play video on background.
strVideoHTML = #"<html><head><style>body.........</style></head> <body><div id=\"overlay\"><div id=\"youtubelogo1\"></div></div><div id=\"player\"></div> <script> var tag = document.createElement('script'); tag.src = \"http://www.youtube.com/player_api\"; var firstScriptTag = document.getElementsByTagName('script')[0]; firstScriptTag.parentNode.insertBefore(tag, firstScriptTag); var player; events: { 'onReady': onPlayerReady, } }); } function onPlayerReady(event) { event.target.playVideo(); } function changeBG(url){ document.getElementById('overlay').style.backgroundImage=url;} function manualPlay() { player.playVideo(); } function manualPause() { player.pauseVideo(); } </script></body> </html>";
NSString *html = [NSString stringWithFormat:strVideoHTML,url, width, height,videoid;
webvideoView.delegate = self;
[webvideoView loadHTMLString:html baseURL:[NSURL URLWithString:#"http://www.your-url.com"]];
On view diappear call
strscript = #"manualPlay();
[webvideoView stringByEvaluatingJavaScriptFromString:strscript];
I'd like to add something that for some reason ended up being the culprit for me. I had used AVPlayer and background play for a long time without problems, but this one time I just couldn't get it to work.
I found out that when you go background, the rate property of the AVPlayer sometimes seems to dip to 0.0 (i.e. paused), and for that reason we simply need to KVO check the rate property at all times, or at least when we go to background. If the rate dips below 0.0 and we can assume that the user wants to play (i.e. the user did not deliberately tap pause in remote controls, the movie ended, etc) we need to call .play() on the AVPlayer again.
AFAIK there is no other toggle on the AVPlayer to keep it from pausing itself when app goes to background.

how to make audio meter level with avaudiorecorder

I am trying to create a audio meter level while I am recording the user voice using avaudiorecorder. Can someone help me in that regard?
Actually, the code is pretty straightforward since AVAudioPlayer and AVAudioRecorder have built in methods to get you on your way. My approach was this:
make a repeating call to -updateMeters and the averagePowerForChannel: & peakPowerForChannel: methods and call a delegate method to notify the controlling object
Example:
NSOperationQueue *queue=[[NSOperationQueue alloc] init];
NSInvocationOperation *operation=[[NSInvocationOperation alloc] initWithTarget:self selector:#selector(updateMeter) object:nil];
[queue addOperation: operation];
and
-(void)updateMeter
{
do {
//don't forget:
[recorder updateMeters];
self.averagePower = [recorder averagePowerForChannel:0];
self.peakPower = [recorder peakPowerForChannel:0];
// we don't want to surprise a ViewController with a method call
// not in the main thread
[self.delegate performSelectorOnMainThread: #selector(meterLevelsDidUpdate:) withObject:self waitUntilDone:NO];
[NSThread sleepForTimeInterval:.05]; // 20 FPS
}while(someCondition);
}
If your View Controller implements the meterLevelsDidUpdate: method, you can use this method to update your Level Meter.
create a UIView subclass with a subview that changes its height according to the average or peak value(s). Adjust to taste.
Easy, you can use NSTimer for that:
- (void)startAudioMetering {
self.meterTimer = [NSTimer scheduledTimerWithTimeInterval:0.1 target:self selector:#selector(updateAudioMeter)userInfo:nil repeats:YES];
}
- (void)stopAudioMetering {
[self.meterTimer invalidate];
}
- (void)updateAudioMeter { //called by timer
// audioRecorder being your instance of AVAudioRecorder
[self.audioRecorder updateMeters];
self.dBLevel = [self.audioRecorder averagePowerForChannel:0];
}
WARNING: While creating your AVAudioRecorder instance, you have to call meteringEnabled AFTER you call prepareToRecord or record, otherwise it won't updateMeters:
[self.audioRecorder prepareToRecord];
self.audioRecorder.meteringEnabled = YES;
Swift code based on Tom's answer:
NSOperationQueue().addOperationWithBlock({[weak self] in
repeat {
self?.audioRecorder.updateMeters()
self?.averagePower = self?.audioRecorder.averagePowerForChannel(0)
self?.peakPower = self?.audioRecorder.peakPowerForChannel(0)
self?.performSelectorOnMainThread(#selector(DictaphoneViewController.updateMeter), withObject: self, waitUntilDone: false)
NSThread.sleepForTimeInterval(0.05)//20 FPS
}
while (someCondition)
})
Do the meter UI stuff inside func updateMeter(){//UI stuff here}
Its pretty simple,
The values you get in the buffer are positive and negative (this is how the waves work) so if you do the average of that values it will give you a near 0 value.
So what you have to do is just put all values positive (with the Math.abs() function) and then do the avarage, it will return you the sound level.
Hope this helps ;)
You can also use ReactiveCocoa and make use of interval:onScheduler::
Returns a signal that sends the current date/time every interval on scheduler.
Using a single audio channel:
#weakify(self)
RACDisposable *metersDisposable = [[RACSignal // make sure you dispose it eventually
interval:0.1
onScheduler:[RACScheduler scheduler]]
subscribeNext:^(NSDate *) {
#strongify(self)
[recorder updateMeters];
auto averagePower = [recorder averagePowerForChannel:0];
auto peakPower = [recorder peakPowerForChannel:0];
// Inform the delegate or make other use of averagePower and peakPower
}];
I found the answer in following link.
AVMeter for AVPlayer
Though it requires lot of customizations but I feel I will be able to do it.

Sound playback delay (hiccup) when OpenAL is implemented on iphone

I implemented OpenAL code to my iphone game. When I starts the game, it runs for 1 sec and stalls for 2 sec then resumes (hiccup effect). I believe its delayed due to the sound files loading. What is the solution? Can anyone recommend any book, site or sources code (not the iphone reference, please)? Is there a loading process and where should I initialize the loading process? Would that help?
Below, I have included the related components of the OpenAL code that I have implemented. The sound file will be played and invoked by a "if" statement in the gameloop. The OpenALController class is for the sound sources and buffers creation and the InitOpenAL method is invoked in OpenALController. MyView is a customized subclass of UIView and connected to the main view (I didn't use the default view).
// MyView.m
// A customized UIView as the main view.
#import "OpenALSoundController.h"
- (void)startPlaying{
...
[self initializeValuables];
...
[self initializeTimer];
}
- (void)initializeTimer {
if (theTimer == nil) {
theTimer = [CADisplayLink displayLinkWithTarget:self selector:#selector)gameLoop)];
theTimer.frameInterval = 2;
[theTimer addToRunLoop: [NSRunLoop currentRunLoop] forMode: NSDefaultRunLoopMode];
}
}
- (void)gameLoop {
...
If something = true
// Play the sound
[[OpenALSoundController sharedSoundController] playSound1];
...
}
...
#end
// OpenALSoundController.h
#interface OpenALSoundController : NSObject {
...}
...
+ (OpenALSoundController*) sharedSoundController
...
#end
// OpenALSoundController.m
// Singleton accessor
{
static OpenALSoundController* shared_sound_controller;
#synchronized(self)
{
if(nil == shared_sound_controller)
{
shared_sound_controller = [[OpenALSoundController alloc] init];
}
return shared_sound_controller;
}
return shared_sound_controller;
}
- (void) initOpenAL{
...
file_url = [[NSURL alloc] initFileURLWithPath:[[NSBundle mainBundle] pathForResource:#"fire" ofType:#"wav"]];
firePcmData = MyGetOpenALAudioDataAll((CFURLRef)file_url, &data_size, &al_format,&sample_rate);
alBufferData(fireOutputBuffer, al_format, firePcmData, data_size, sample_rate);
[file_url release];
...
alSourcei(outputSourceFire, AL_BUFFER, fireOutputBuffer);
...
}
You might be interested in Finch, an OpenAL sound engine for iOS. It’s very well suited to games. It’s usually better to reuse some already existing code than develop and maintain your own.
First its better to use mp3, as wav files are huge and loading from disk takes time. Mp3 files are smaller on disk, loaded into memory and decompressed there for playing. Try experimenting by reducing mp3 bitrate/encoding quality too.
Also you need to preload sounds to avoid hiccups, otherwise you will have a delay the first time a sound is played.

Play sound with screen turned off / don't let iPhone go to sleep

So I have an application and I want to keep it working even if the screen is turned off.
Previously when I wanted to do that I used this hack/trick - I play a silent/empty sound in a loop in the background (AudioServicesPlaySystemSound) so if user presses the on/off button the application still works in the background - so it never allow iPhone to go to sleep mode - it just turned off the screen and maybe things like wifi or bluetooth (and on the iPod Touch accelerometers as far as I remember). And it worked. I wanted to use the same trick in my new application but when I was testing it now it seems it doesn't work anymore. The sound in the background plays (when I replace "empty" audio file with some sound I can hear it play) even with screen turned off but the sound it should play (using AVAudioPlayer) doesn't play (even when I turn the screen on again).
I don't know at which point it stopped working (it worked on 3.x OS for sure). Am I doing something wrong? Did Apple changed/fixed the "hack" that allowed your app to work even with screen turned off? Is there another way to allow the device to go to sleep (and drain the battery less) but continue to work?
This is the code I use to play background/silent sound:
-(void) playSilentSound
{
CFBundleRef mainBundle = CFBundleGetMainBundle ();
CFURLRef silentUrl = CFBundleCopyResourceURL (mainBundle, CFSTR ("silence"), CFSTR ("aiff"), NULL);
AudioServicesCreateSystemSoundID (silentUrl, &silentSound);
silentTimer = [NSTimer scheduledTimerWithTimeInterval: 2.0 target: self selector:#selector(playSilence) userInfo: nil repeats: YES];
}
-(void) playSilence
{
AudioServicesPlaySystemSound (silentSound);
}
And this is how I play the sound that should play even if the screen is turned off:
-(BOOL) playSound: (NSString *) path withLoops: (BOOL) loops stopAfter: (int) seconds
{
NSError *error;
player = [[AVAudioPlayer alloc] initWithContentsOfURL: [NSURL fileURLWithPath: path] error: &error];
player.delegate = self;
player.numberOfLoops = 0;
player.volume = volume;
secondsPlayed = 0;
loop = loops;
BOOL played = [player play];
if(played && seconds > 0)
{
timer = [[NSTimer scheduledTimerWithTimeInterval: 0.5 target: self selector: #selector(stopPlaying:) userInfo: [NSNumber numberWithInt: seconds] repeats: YES] retain];
secondsLimit = seconds;
} else {
secondsLimit = -1;
}
}
-(void) stopPlaying:(NSTimer*)theTimer
{
if(secondsLimit > 0 && (secondsPlayed + [player currentTime]) >= secondsLimit)
{
[player stop];
[timer release];
timer = nil;
}
}
- (void)audioPlayerDidFinishPlaying:(AVAudioPlayer *)sender successfully:(BOOL)flag
{
if(sender == player)
{
if(flag) { secondsPlayed += sender.duration; }
if(loop) { [player play]; }
}
}
The code is a bit complicated maybe - it could be just the first few lines - it's made that way so you can play only X seconds of sound (if sound is shorter than X it will play few loops until the total time is >= X). And of course everything is working fine when the screen is left on.
Also - if you find the code useful in your projects (like playSound:withLoops:stopAfter:) - feel free to use it (but it would be cool if you send me a message so I would know that I helped :)).
So the answer is to set session category. Some categories just turn off sound when the device is being locked. For me the best option was AVAudioSessionCategoryPlayback
[[AVAudioSession sharedInstance] setCategory: AVAudioSessionCategoryPlayback error: nil];
I don't know about Apple fixing stuff, but in iOS 4 there are now official ways to continue doing things in the background which you should take advantage of.

AVAudioPlayer is not rentrant correct?

It appears that AVAudioPlayer is not reentent. So in the following would soundfx play to completion, delay 1 second, then play again, rather then the 1 second delay - and resultant overlapping playback I desire:
// ...
AVAudioPlayer* soundfx = [[AVAudioPlayer alloc] initWithContentsOfURL:soundURL error:nil];
(void) makeSomeNoise {
[soundfx play];
sleep(1);
[soundfx play];
}
Must I then resort to NSOperation - threading - to achieve my goal of overlapping playback?
Note: I am using IMA4/ADPCM format which is apparently the correct format for layered sound playback.
Cheers,
Doug
It's not that AVAudioPlayer is not reentrant. It is that AVAudioPlayer starts to play your sound after the runloop has ended, and not in your makeSomeNoise function. If you want to play your sound with a one second delay, you can do this:
(void) makeSomeNoise {
[soundfx play];
[soundfx performSelector:play withObject: nil afterDelay:1.0];
}