iOS SDK 4.3 OpenAL alGenSources results in AL_INVALID_OPERATION - iphone

I'm trying to get to grips with OpenAL, working through a tutorial here: http://benbritten.com/2008/11/06/openal-sound-on-the-iphone/
My problem is that the sound does not play, although there are no iOS errors thrown. There is an OpenAL error though. The code sample below is the body of an IBAction method, and results in an AL_INVALID_OPERATION at alGenSources(1, &sourceID). sourceID reports as NULL.
I've tried this on the device and the simulator.
This code sample seems to be in pretty wide use, but I can't find anybody complaining of this particular problem. Can anybody throw any light on this? Many thanks for any help,
NSString *audioFileName = [[NSBundle mainBundle] pathForResource:#"1" ofType:#"caf"];
AudioFileID fileID = [self openAudioFile:audioFileName];
UInt32 filesize = [self audioFileSize:fileID];
unsigned char *outData = malloc(filesize);
OSStatus result = noErr;
result = AudioFileReadBytes(fileID, false, 0, &filesize, outData);
AudioFileClose(fileID);
if (result != 0) {
NSLog(#"Can't load file..");
}
NSUInteger bufferID;
//NSLog(#"bufferID %#", [NSNumber numberWithUnsignedInteger:bufferID]);
alGenBuffers(1, &bufferID);
//NSLog(#"bufferID %#", [NSNumber numberWithUnsignedInteger:bufferID]);
alBufferData(bufferID, AL_FORMAT_STEREO16, outData, filesize, 44100);
[bufferStorageArray addObject:[NSNumber numberWithUnsignedInteger:bufferID]];
alGetError();
ALuint sourceID;
alGenSources(1, &sourceID);
if(alGetError() == AL_INVALID_OPERATION)
{
printf("\n++++ Error creating buffers INVALID_OPERATION!!\n");
//exit(1);
}
else
{
printf("No errors yet.");
}
alSourcei(sourceID, AL_BUFFER, bufferID);
alSourcef(sourceID, AL_PITCH, 1.0f);
alSourcef(sourceID, AL_GAIN, 1.0f);
if (loops) {
alSourcei(sourceID, AL_LOOPING, AL_TRUE);
}
[soundDictionary setObject:[NSNumber numberWithUnsignedInt:sourceID] forKey:#"sound"];
if (outData) {
free(outData);
outData = NULL;
}
[self playSound:#"sound"];

For your pitch problem, make sure the sound file you are loading matches the sample rate you are feeding into alBufferData. Your caf file is probably saved at 22050 Hz.
AudioStreamBasicDescription's mSampleRate will tell you what the audio file's sample rate really is.
You should also check mChannelsPerFrame to make sure it really is stereo sound.
Also, OpenAL by default on iOS only generates 4 stereo sources. If you try to load more than 4 sources with stereo data, your audio will sound like garbage. You can change that by specifying attributes ALC_STEREO_SOURCES and ALC_MONO_SOURCES when you create a context. You have a maximum of 32 sources (by default it sets up 28 mono and 4 stereo sources).

Stupid mistake on my part - I had initialised OpenAL in initWithNibName, which was never being called. Moving the init into viewDidLoad has got everything working, although playback is chipmunk-style high speed

Related

Getting Potential Leak of an object in CVPixelBuffer

i am creating a app to screen capture from the iphone. So after i did the coding i used profiling and analyzing to check memory leaks. I am getting only one memory leak in one section in the code. Here is my code which gives me the memory leak.
-(void) writeSample: (NSTimer*) _timer {
if (assetWriterInput.readyForMoreMediaData) {
// CMSampleBufferRef sample = nil;
CVReturn cvErr = kCVReturnSuccess;
// get screenshot image!
CGImageRef image = (CGImageRef) [[self screenshot] CGImage];
NSLog (#"made screenshot");
// prepare the pixel buffer
CVPixelBufferRef pixelBuffer = NULL;
CFDataRef imageData= CGDataProviderCopyData(CGImageGetDataProvider(image));
NSLog (#"copied image data");
cvErr = CVPixelBufferCreateWithBytes(kCFAllocatorDefault,
FRAME_WIDTH,
FRAME_HEIGHT,
kCVPixelFormatType_32BGRA,
(void*)CFDataGetBytePtr(imageData),
CGImageGetBytesPerRow(image),
NULL,
NULL,
NULL,
&pixelBuffer);
NSLog (#"CVPixelBufferCreateWithBytes returned %d", cvErr);
// calculate the time
CFAbsoluteTime thisFrameWallClockTime = CFAbsoluteTimeGetCurrent();
CFTimeInterval elapsedTime = thisFrameWallClockTime - firstFrameWallClockTime;
NSLog (#"elapsedTime: %f", elapsedTime);
CMTime presentationTime = CMTimeMake (elapsedTime * TIME_SCALE, TIME_SCALE);
// write the sample
BOOL appended = [assetWriterPixelBufferAdaptor appendPixelBuffer:pixelBuffer withPresentationTime:presentationTime];
if (appended) {
NSLog (#"appended sample at time %lf", CMTimeGetSeconds(presentationTime));
} else {
NSLog (#"failed to append");
[self stopRecording];
self.startStopButton.selected = NO;
}
}
}
it says Potential leak of an object stored into 'imageData'. Can any one help me with finding the error in the above code. There is a memory leak in above code when i check it with the memory management tools too. If any one can help me it would be a great help.
Thanks in Advance !!
From comments -
Do a CFRelease on your imageData when your done with it?
You can put it right before or right after NSLog (#"CVPixelBufferCreateWithBytes returned %d", cvErr);
CFRelease(imageData);
http://developer.apple.com/library/mac/#documentation/QuartzCore/Reference/CVPixelBufferRef/Reference/reference.html
I am not sure about the rest of the code you have, but generally when there is a call with Crete as a word in it, it has to have a corresponding release statement. Please check the documentation above.
CVPixelBufferRelease
Releases a pixel buffer.
void CVPixelBufferRelease (
CVPixelBufferRef texture
);

Is it possible to increase audio volume on system sound?

I am making piano application for iPad. I am using AudioServicesPlaySystemSound (toneSSID) for play sound of keys. But there also functionality to increase volume. But I not find how to increase sound. Can anybody help me?
Thanks.
Add AudioStreamer.h file
Add this method to AudioStreamer.m
- (void)setVolume:(float)Level
{
OSStatus errorMsg = AudioQueueSetParameter(audioQueue, kAudioQueueParam_Volume, Level);
if (errorMsg) {
NSLog(#"AudioQueueSetParameter returned %d when setting the volume.", errorMsg);
}
}
And handle this to audiostreamclass object like
AudioStreamer *Obj = [AudioStreamer alloc] init];
[obj setVolume:1.0];

Hardware accelerated h.264 decoding to texture, overlay or similar in iOS

Is it possible, and supported, to use the iOS hardware accelerated h.264 decoding API to decode a local (not streamed) video file, and then compose other objects on top of it?
I would like to make an application that involves drawing graphical objects in front of a video, and use the playback timer to synchronize what I am drawing on top, to what is being played on the video. Then, based on the user's actions, change what I am drawing on top (but not the video)
Coming from DirectX, OpenGL and OpenGL ES for Android, I am picturing something like rendering the video to a texture, and using that texture to draw a full screen quad, then use other sprites to draw the rest of the objects; or maybe writing an intermediate filter just before the renderer, so I can manipulate the individual output frames and draw my stuff; or maybe drawing to a 2D layer on top of the video.
It seems like AV Foundation, or Core Media may help me do what I am doing, but before I dig into the details, I would like to know if it is possible at all to do what I want to do, and what are my main routes to approach the problem.
Please refrain from "this is too advanced for you, try hello world first" answers. I know my stuff, and just want to know if what I want to do is possible (and most importantly, supported, so the app won't get eventually rejected), before I study the details by myself.
edit:
I am not knowledgeable in iOS development, but professionally do DirectX, OpenGL and OpenGL ES for Android. I am considering making an iOS version of an Android application I currently have, and I just want to know if this is possible. If so, I have enough time to start iOS development from scratch, up to doing what I want to do. If it is not possible, then I will just not invest time studying the entire platform at this time.
Therefore, this is a technical feasibility question. I am not requesting code. I am looking for answers of the type "Yes, you can do that. Just use A and B, use C to render into D and draw your stuff with E", or "No, you can't. The hardware accelerated decoding is not available for third-party applications" (which is what a friend told me). Just this, and I'll be on my way.
I have read the overview for the video technologies in page 32 of the ios technology overview. It pretty much says that I can use Media Player for the most simple playback functionality (not what I'm looking for), UIKit for embedding videos with a little more control over the embedding, but not over the actual playback (not what I'm looking for), AVFoundation for more control over playback (maybe what I need, but most of the resources I find online talk about how to use the camera), or Core Media to have full low-level control over video (probably what I need, but extremely poorly documented, and even more lacking in resources on playback than even AVFoundation).
I am concerned that I may dedicate the next six months to learn iOS programming full time, only to find at the end that the relevant API is not available for third party developers, and what I want to do is unacceptable for iTunes store deployment. This is what my friend told me, but I can't seem to find anything relevant in the app development guidelines. Therefore, I came here to ask people who have more experience in this area, whether or not what I want to do is possible. No more.
I consider this a valid high level question, which can be misunderstood as an I-didn't-do-my-homework-plz-give-me-teh-codez question. If my judgement in here was mistaken, feel free to delete, or downvote this question to your heart's contempt.
Yes, you can do this, and I think your question was specific enough to belong here. You're not the only one who has wanted to do this, and it does take a little digging to figure out what you can and can't do.
AV Foundation lets you do hardware-accelerated decoding of H.264 videos using an AVAssetReader, at which point you're handed the raw decoded frames of video in BGRA format. These can be uploaded to a texture using either glTexImage2D() or the more efficient texture caches in iOS 5.0. From there, you can process for display or retrieve the frames from OpenGL ES and use an AVAssetWriter to perform hardware-accelerated H.264 encoding of the result. All of this uses public APIs, so at no point do you get anywhere near something that would lead to a rejection from the App Store.
However, you don't have to roll your own implementation of this. My BSD-licensed open source framework GPUImage encapsulates these operations and handles all of this for you. You create a GPUImageMovie instance for your input H.264 movie, attach filters onto it (such as overlay blends or chroma keying operations), and then attach these filters to a GPUImageView for display and/or a GPUImageMovieWriter to re-encode an H.264 movie from the processed video.
The one issue I currently have is that I don't obey the timestamps in the video for playback, so frames are processed as quickly as they are decoded from the movie. For filtering and re-encoding of a video, this isn't a problem, because the timestamps are passed through to the recorder, but for direct display to the screen this means that the video can be sped up by as much as 2-4X. I'd welcome any contributions that would let you synchronize the playback rate to the actual video timestamps.
I can currently play back, filter, and re-encode 640x480 video at well over 30 FPS on an iPhone 4 and 720p video at ~20-25 FPS, with the iPhone 4S being capable of 1080p filtering and encoding at significantly higher than 30 FPS. Some of the more expensive filters can tax the GPU and slow this down a bit, but most filters operate in these framerate ranges.
If you want, you can examine the GPUImageMovie class to see how it does this uploading to OpenGL ES, but the relevant code is as follows:
- (void)startProcessing;
{
NSDictionary *inputOptions = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] forKey:AVURLAssetPreferPreciseDurationAndTimingKey];
AVURLAsset *inputAsset = [[AVURLAsset alloc] initWithURL:self.url options:inputOptions];
[inputAsset loadValuesAsynchronouslyForKeys:[NSArray arrayWithObject:#"tracks"] completionHandler: ^{
NSError *error = nil;
AVKeyValueStatus tracksStatus = [inputAsset statusOfValueForKey:#"tracks" error:&error];
if (!tracksStatus == AVKeyValueStatusLoaded)
{
return;
}
reader = [AVAssetReader assetReaderWithAsset:inputAsset error:&error];
NSMutableDictionary *outputSettings = [NSMutableDictionary dictionary];
[outputSettings setObject: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey: (NSString*)kCVPixelBufferPixelFormatTypeKey];
// Maybe set alwaysCopiesSampleData to NO on iOS 5.0 for faster video decoding
AVAssetReaderTrackOutput *readerVideoTrackOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:[[inputAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] outputSettings:outputSettings];
[reader addOutput:readerVideoTrackOutput];
NSArray *audioTracks = [inputAsset tracksWithMediaType:AVMediaTypeAudio];
BOOL shouldRecordAudioTrack = (([audioTracks count] > 0) && (self.audioEncodingTarget != nil) );
AVAssetReaderTrackOutput *readerAudioTrackOutput = nil;
if (shouldRecordAudioTrack)
{
audioEncodingIsFinished = NO;
// This might need to be extended to handle movies with more than one audio track
AVAssetTrack* audioTrack = [audioTracks objectAtIndex:0];
readerAudioTrackOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:nil];
[reader addOutput:readerAudioTrackOutput];
}
if ([reader startReading] == NO)
{
NSLog(#"Error reading from file at URL: %#", self.url);
return;
}
if (synchronizedMovieWriter != nil)
{
__unsafe_unretained GPUImageMovie *weakSelf = self;
[synchronizedMovieWriter setVideoInputReadyCallback:^{
[weakSelf readNextVideoFrameFromOutput:readerVideoTrackOutput];
}];
[synchronizedMovieWriter setAudioInputReadyCallback:^{
[weakSelf readNextAudioSampleFromOutput:readerAudioTrackOutput];
}];
[synchronizedMovieWriter enableSynchronizationCallbacks];
}
else
{
while (reader.status == AVAssetReaderStatusReading)
{
[self readNextVideoFrameFromOutput:readerVideoTrackOutput];
if ( (shouldRecordAudioTrack) && (!audioEncodingIsFinished) )
{
[self readNextAudioSampleFromOutput:readerAudioTrackOutput];
}
}
if (reader.status == AVAssetWriterStatusCompleted) {
[self endProcessing];
}
}
}];
}
- (void)readNextVideoFrameFromOutput:(AVAssetReaderTrackOutput *)readerVideoTrackOutput;
{
if (reader.status == AVAssetReaderStatusReading)
{
CMSampleBufferRef sampleBufferRef = [readerVideoTrackOutput copyNextSampleBuffer];
if (sampleBufferRef)
{
runOnMainQueueWithoutDeadlocking(^{
[self processMovieFrame:sampleBufferRef];
});
CMSampleBufferInvalidate(sampleBufferRef);
CFRelease(sampleBufferRef);
}
else
{
videoEncodingIsFinished = YES;
[self endProcessing];
}
}
else if (synchronizedMovieWriter != nil)
{
if (reader.status == AVAssetWriterStatusCompleted)
{
[self endProcessing];
}
}
}
- (void)processMovieFrame:(CMSampleBufferRef)movieSampleBuffer;
{
CMTime currentSampleTime = CMSampleBufferGetOutputPresentationTimeStamp(movieSampleBuffer);
CVImageBufferRef movieFrame = CMSampleBufferGetImageBuffer(movieSampleBuffer);
int bufferHeight = CVPixelBufferGetHeight(movieFrame);
int bufferWidth = CVPixelBufferGetWidth(movieFrame);
CFAbsoluteTime startTime = CFAbsoluteTimeGetCurrent();
if ([GPUImageOpenGLESContext supportsFastTextureUpload])
{
CVPixelBufferLockBaseAddress(movieFrame, 0);
[GPUImageOpenGLESContext useImageProcessingContext];
CVOpenGLESTextureRef texture = NULL;
CVReturn err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault, coreVideoTextureCache, movieFrame, NULL, GL_TEXTURE_2D, GL_RGBA, bufferWidth, bufferHeight, GL_BGRA, GL_UNSIGNED_BYTE, 0, &texture);
if (!texture || err) {
NSLog(#"Movie CVOpenGLESTextureCacheCreateTextureFromImage failed (error: %d)", err);
return;
}
outputTexture = CVOpenGLESTextureGetName(texture);
// glBindTexture(CVOpenGLESTextureGetTarget(texture), outputTexture);
glBindTexture(GL_TEXTURE_2D, outputTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
for (id<GPUImageInput> currentTarget in targets)
{
NSInteger indexOfObject = [targets indexOfObject:currentTarget];
NSInteger targetTextureIndex = [[targetTextureIndices objectAtIndex:indexOfObject] integerValue];
[currentTarget setInputSize:CGSizeMake(bufferWidth, bufferHeight) atIndex:targetTextureIndex];
[currentTarget setInputTexture:outputTexture atIndex:targetTextureIndex];
[currentTarget newFrameReadyAtTime:currentSampleTime];
}
CVPixelBufferUnlockBaseAddress(movieFrame, 0);
// Flush the CVOpenGLESTexture cache and release the texture
CVOpenGLESTextureCacheFlush(coreVideoTextureCache, 0);
CFRelease(texture);
outputTexture = 0;
}
else
{
// Upload to texture
CVPixelBufferLockBaseAddress(movieFrame, 0);
glBindTexture(GL_TEXTURE_2D, outputTexture);
// Using BGRA extension to pull in video frame data directly
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, bufferWidth, bufferHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(movieFrame));
CGSize currentSize = CGSizeMake(bufferWidth, bufferHeight);
for (id<GPUImageInput> currentTarget in targets)
{
NSInteger indexOfObject = [targets indexOfObject:currentTarget];
NSInteger targetTextureIndex = [[targetTextureIndices objectAtIndex:indexOfObject] integerValue];
[currentTarget setInputSize:currentSize atIndex:targetTextureIndex];
[currentTarget newFrameReadyAtTime:currentSampleTime];
}
CVPixelBufferUnlockBaseAddress(movieFrame, 0);
}
if (_runBenchmark)
{
CFAbsoluteTime currentFrameTime = (CFAbsoluteTimeGetCurrent() - startTime);
NSLog(#"Current frame time : %f ms", 1000.0 * currentFrameTime);
}
}

How do you force audio to come out of the iPhone speaker when a headset is attached?

I'm trying to add a loudspeaker feature to one of my iPhone apps. I already created the recording functionality, but when I play the recorded audio it only plays to the phone headset.
What I need is the recorded file to be played on the loudspeaker, even if there is a headset attached. How could I reroute the audio to do this?
You need to override the default audio properties using AudioSessionSetProperty. Look at something like this to force all audio to go to the speaker (note that this will even happen if headphones are plugged in).
OSStatus err = 0;
UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
err = AudioSessionSetProperty(kAudioSessionProperty_OverrideAudioRoute,sizeof(audioRouteOverride),&audioRouteOverride);
To detect the headphones, try this (this is literally copy/paste code off of another SO post, so caveat emptor, but it works for me):
/**
* Tells us if the headset is plugged in
*/
- (BOOL) headsetIsPluggedIn
{
BOOL returnVal = NO;
UInt32 routeSize = sizeof(CFStringRef);
CFStringRef route = NULL;
OSStatus error = AudioSessionGetProperty(kAudioSessionProperty_AudioRoute, &routeSize, &route);
if (!error && (route != NULL) && ([(NSString*)route rangeOfString:#"Head"].location != NSNotFound))
{
CFRelease(route);
returnVal = YES;
}
return returnVal;
}
EDIT: There is a bit of a discussion in the comments about whether the CFRelease is appropriate or not. Any hardcore Core Foundation experts care to weigh in?

Weird popping noise when playing different sounds with different volumes set through OpenAL on the iPhone

I'm using OpenAL sound framework on the iPhone, and I'm setting different volumes on individual sounds. I'm running into a problem where I'm hearing an initial popping/clicking noise when switching from one sound to the next.
It's really noticeable when I have one sound that's got a high volume (1.0) and a second
sound that has a low one (0.2). When I hit the loud sound, and then
hit the soft sound, I hear the pop/click. But when I go from the soft
sound to the loud, I don't notice anything. So the pop/click really
happens when switching from loud to soft sounds.
Here's the init sound method:
- (id) initWithSoundFile:(NSString *)file doesLoop:(BOOL)loops
{
self = [super init];
if (self != nil)
{
if(![self loadSoundFile:file doesLoop:loops])
{
debug(#"Failed to load the sound file: %#...", file);
[self release];
return nil;
}
self.sourceFileName = file;
//temporary sound queue
self.temporarySounds = [NSMutableArray array];
//default volume/pitch
self.volume = 1.0;
self.pitch = 1.0;
}
return self;
}
and here's the play function:
- (BOOL) play
{
if([self isPlaying]) //see if the base source is busy...
{
//if so, create a new source
NSUInteger tmpSourceID;
alGenSources(1, &tmpSourceID);
//attach the buffer to the source
alSourcei(tmpSourceID, AL_BUFFER, bufferID);
alSourcePlay(tmpSourceID);
//add the sound id to the play queue so we can dispose of it later
[temporarySounds addObject: [NSNumber numberWithUnsignedInteger:tmpSourceID]];
//a "callback" for when the sound is done playing +0.1 secs
[self performSelector:#selector(deleteTemporarySource)
withObject:nil
afterDelay:(duration * pitch) + 0.1];
return ((error = alGetError()) != AL_NO_ERROR);
}
//if the base source isn't busy, just use that one...
alSourcePlay(sourceID);
return ((error = alGetError()) != AL_NO_ERROR);
}
and here's the function where i set the volume for each sound immediately after playing (ive tried setting it before playing too):
- (void) setVolume:(ALfloat)newVolume
{
volume = MAX(MIN(newVolume, 1.0f), 0.0f); //cap to 0-1
alSourcef(sourceID, AL_GAIN, volume);
//now set the volume for any temporary sounds...
for(NSNumber *tmpSourceID in temporarySounds)
{
//tmpSourceID is the source ID for the temporary sound
alSourcef([tmpSourceID unsignedIntegerValue], AL_GAIN, volume);
}
}
Any help is greatly appreciated as I've tried everything I can think of. I would be so grateful.
All I had to do was use calloc instead of malloc to allocate memory for the OpenAL buffer.
Or you could also zero set the memory with memset.
The wierd popping noise went off. It was due to junk memory in my case. That's why it was random too. Hope this helps.
This problem is caused by not calling alSourceStop.
The documentation doesn't really state this, but alSourceStop must be called on a sound source before it can be reused even if the sound had already finished and the AL_SOURCE_STATE parameter of the source is not AL_PLAYING.
I've randomly got to this unaswered question and, finding that the problem was not solved, I'll try to give my answer, even if a long time has passed.
I don't know OpenAL, but it sounds like this is a purely audio problem. It is normal to hear short clicks when you change suddenly the level of the audio, especially from a high value to a low value. For example, if you map directly the volume of the audio to a slider, which value is updated every few ms, you can easily hear clicks and pops when sliding fast the control. What audio software developers do is smoothing the parameter changes with a low pass filter.
In your case, I would suggest you to stop the clip after fading it out, and start a new clip by fading it in. The fade time can be as short as 2 ms: it's not audible, and the sound will play just finely.
I wonder if (some versions of) OpenAL can automatically deal with this issue.