I'm working on an app for the ipad 2 that lets the user record video of himself using the device front camera, and then play it back on a video player. I have the overall functionality working, but sometimes, just sometimes my app crashes when I load the view where the video will be played back, because of this:
'CALayerInvalidGeometry', reason: 'CALayer position contains NaN: [nan 11.5]'
I have noticed that the app crashes mostly, but not exclusively, when the recorded clip playing is less than around 15 seconds long.
Anyone have an idea whats going on?
Heres the code that takes care of recording:
-(void)record{
AVCaptureMovieFileOutput *output = [[AVCaptureMovieFileOutput alloc]init];
NSMutableString *videoURL;
if(isRecording){
//here i do some stuff to generate a random system path
[session addOutput:output];
AVCaptureConnection *videoConnection;
[session beginConfiguration];
for ( AVCaptureConnection *connection in [output connections] ) {
for ( AVCaptureInputPort *port in [connection inputPorts] ) {
if ( [[port mediaType] isEqual:AVMediaTypeVideo] ) {
videoConnection = connection;
}
}
}
if([videoConnection isVideoOrientationSupported]){
[videoConnection setVideoOrientation:AVCaptureVideoOrientationLandscapeLeft];
videoConnection.videoMirrored = true;
}
[session commitConfiguration];
[session startRunning];
NSURL *vidURL = [[NSURL alloc]initFileURLWithPath:videoURL];
[output startRecordingToOutputFileURL:vidURL recordingDelegate:self];
NSLog(#"Recording started in %#", videoURL);
[rootRep addObject:videoURL];
[vidURL release];
[videoURL release];
}else{
isRecording = false;
[output stopRecording];
[session removeOutput:output];
[output release];
NSLog(#"Recording stopped");
[recBut setImage:[UIImage imageNamed:#"rec.png"] forState:UIControlStateNormal];
}
}
EDIT: I've implemented a method to analize all the captured videos and delete the faulty ones, my app is stable again, but I still wonder why some videos don't get created well.
I don't know why exactly the camera doesn't capture good video files every time, but I'll show you how you can safely test whether the file works or not. You have to create an AVAsset with the file path you want to test, the AVAsset has a property called 'playable', a bool, then you can cycle an array with the addresses of your captured videos asking if the current asset is playable and deleting the corrupted files.
Related
I am writing an application to show stats on the light conditions as seen by the iphone camera. I take an image every second, and the performing calculations on it.
To capture an image, I am using the following method:
-(void) captureNow
{
AVCaptureConnection *videoConnection = nil;
for (AVCaptureConnection *connection in captureManager.stillImageOutput.connections)
{
for (AVCaptureInputPort *port in [connection inputPorts])
{
if ([[port mediaType] isEqual:AVMediaTypeVideo] )
{
videoConnection = connection;
break;
}
}
if (videoConnection) { break; }
}
[captureManager.stillImageOutput captureStillImageAsynchronouslyFromConnection:videoConnection completionHandler: ^(CMSampleBufferRef imageSampleBuffer, NSError *error)
{
NSData *imageData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageSampleBuffer];
latestImage = [[UIImage alloc] initWithData:imageData];
}];
}
However, the captureStillImageAsynhronously.... method causes the 'shutter' sound to be played by the phone, which is no good for my application, as it will be capturing images constantly.
I have read that it is not possible to disable this sound effect. Instead, I want to capture frames from the video input for the phone:
AVCaptureDeviceInput *newVideoInput = [[AVCaptureDeviceInput alloc] initWithDevice:[self backFacingCamera] error:nil];
and hopefully turn these into UIImage objects.
How would I achieve this? I don't know that much about how the AVFoundation stuff is working - I downloaded some example code and modified it for my purposes.
Don't use a still camera for this. Instead, grab from the video camera of the device and process the data contained within the pixel buffer you get in response to being an AVCaptureVideoDataOutputSampleBufferDelegate.
You can set up a video connection using code like the following:
// Grab the back-facing camera
AVCaptureDevice *backFacingCamera = nil;
NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
for (AVCaptureDevice *device in devices)
{
if ([device position] == AVCaptureDevicePositionBack)
{
backFacingCamera = device;
}
}
// Create the capture session
captureSession = [[AVCaptureSession alloc] init];
// Add the video input
NSError *error = nil;
videoInput = [[[AVCaptureDeviceInput alloc] initWithDevice:backFacingCamera error:&error] autorelease];
if ([captureSession canAddInput:videoInput])
{
[captureSession addInput:videoInput];
}
// Add the video frame output
videoOutput = [[AVCaptureVideoDataOutput alloc] init];
[videoOutput setAlwaysDiscardsLateVideoFrames:YES];
[videoOutput setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
[videoOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()];
if ([captureSession canAddOutput:videoOutput])
{
[captureSession addOutput:videoOutput];
}
else
{
NSLog(#"Couldn't add video output");
}
// Start capturing
[captureSession setSessionPreset:AVCaptureSessionPreset640x480];
if (![captureSession isRunning])
{
[captureSession startRunning];
};
You'll then need to process these frames in a delegate method that looks like the following:
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
CVImageBufferRef cameraFrame = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(cameraFrame, 0);
int bufferHeight = CVPixelBufferGetHeight(cameraFrame);
int bufferWidth = CVPixelBufferGetWidth(cameraFrame);
// Process pixel buffer bytes here
CVPixelBufferUnlockBaseAddress(cameraFrame, 0);
}
The raw pixel bytes for your BGRA image will be contained within the array starting at CVPixelBufferGetBaseAddress(cameraFrame). You can iterate over those to obtain your desired values.
However, you'll find that any operation performed over the entire image on the CPU will be a little slow. You can use Accelerate to help with an average color operation, like you want here. I've used vDSP_meanv() in the past to average luminance values, once you have those in an array. For something like that, you might be best served to grab YUV planar data from the camera instead of the BGRA values I pull down here.
I've also written an open source framework to process video using OpenGL ES, although I don't yet have whole-image reduction operations in there like you'd need for the kind of image analysis here. My histogram generator is probably the closest I have to what you're trying to do.
I am using the following code to capture video and save it to the documents folder of my app:
AVCaptureDeviceInput *captureInput = [AVCaptureDeviceInput deviceInputWithDevice:[AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo] error:NULL];
m_captureFileOutput = [[AVCaptureMovieFileOutput alloc] init];
captureSession = [[AVCaptureSession alloc] init];
[captureSession addInput:captureInput];
[captureSession addOutput:m_captureFileOutput];
[captureSession beginConfiguration];
[captureSession setSessionPreset:AVCaptureSessionPresetHigh];
[captureSession commitConfiguration];
[captureSession startRunning];
...some function that starts the recording process...
[m_captureFileOutput startRecordingToOutputFileURL:url recordingDelegate:self];
...some function that ends the recording process...
[m_captureFileOutput stopRecording];
The catch is, my goal is to be able to record up to 9 hours of video at a time. Practically, is is feasible to record a video of this size using this method? Does AVCaptureMovieFileOutput encode and save the video to the disk in real time as it receives frames from the camera, or is the entire video buffered in RAM before being processed after [m_captureFileOutput stopRecording]; is called?
If this approach is not reasonable for recording such a long duration of video, what might be a reasonable alternative?
Thanks,
James
Pretty sure the AVCaptureMovieFileOutput appends to the file and does not use a in memory buffer (maybe it does but it flushes it to the file before it gets too large)...some evidence of this can be seen in the movieFragmentInterval property here. Also i have used this method to record to a file for large files, and it works ok, if it was keeping the file in memory one would run out of memory pretty quickly under some presets (1280x720 for example)
I am writing an application that uses the AVAudioRecorder class. It works great except for when a phone call comes in. I am handling this per apple's guidelines of using the AVAudioRecorderDelegate methods
– (void) audioRecorderBeginInterruption:
– (void) audioRecorderEndInterruption:
It works great until the interruption ends and I attempt to "resume" the recording by calling the record method again (per the documentation). However it does not resume my recording but instead throws out the old one and starts up an entirely new one in its place. I have not been able to find a solution to this problem, if anyone has figured this out, or if it is a bug with apple's AVAudioRecorder please let me know. I really hope I do not have to write this using AudioQueues.
thanks
Looks like its a bug with apple's API. Great fun....
This was the response we received from a support ticket.
"The behavior you described is a bug and unfortunately there's nothing in the API that you can change to work around to actually append to the original recording. The interruption is resulting in capturing only the audio recorded after the interruption. You could try and stop the recording after the interruption then creating a new file after which would at least not cause the user to loose any information, but the result would be two separate files.
Please file a bug report at for this issue since bugs filed by external developers are critical when iOS engineering is evaluating critical features of fixes to address. It's easily reproducible but if you have a test app you can include please do, iOS engineering like having apps that show the bug directly.
"
My solution was:
Start record on temp file
Watch for AVAudioSessionInterruptionNotificatio
On AVAudioSessionInterruptionTypeBegan - stop the recording.
On AVAudioSessionInterruptionTypeEnded - Start new recording.
When the user stops - Marge the files.
Full Code
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(audioSessionInterruptionNotification:)
name:AVAudioSessionInterruptionNotification
object:audioSession];
-(void)audioSessionInterruptionNotification:(NSNotification*)notification {
NSString* seccReason = #"";
//Check the type of notification, especially if you are sending multiple AVAudioSession events here
NSLog(#"Interruption notification name %#", notification.name);
NSError *err = noErr;
if ([notification.name isEqualToString:AVAudioSessionInterruptionNotification]) {
seccReason = #"Interruption notification received";
//Check to see if it was a Begin interruption
if ([[notification.userInfo valueForKey:AVAudioSessionInterruptionTypeKey] isEqualToNumber:[NSNumber numberWithInt:AVAudioSessionInterruptionTypeBegan]]) {
seccReason = #"Interruption began";
NSLog(#"Interruption notification name %# audio pause", notification.name);
dispatch_time_t restartTime = dispatch_time(DISPATCH_TIME_NOW,
0.01 * NSEC_PER_SEC);
dispatch_after(restartTime, dispatch_get_global_queue(0, 0), ^{
AVAudioRecorder *recorder = [[self recorderPool] objectForKey:lastRecID];
if (recorder) {
if(recorder.isRecording) {
[recorder stop];
NSLog(#"Interruption notification name Pauseing recording %#", lastRecID);
} else {
NSLog(#"Interruption notification name Already Paused %#", lastRecID);
}
}else {
NSLog(#"Interruption notification name recording %# not found", lastRecID);
}
NSLog(#"Interruption notification Pauseing recording status %d",recorder.isRecording);
});
} else if([[notification.userInfo valueForKey:AVAudioSessionInterruptionTypeKey] isEqualToNumber:[NSNumber numberWithInt:AVAudioSessionInterruptionTypeEnded]]){
seccReason = #"Interruption ended!";
NSLog(#"Interruption notification name %# audio resume", notification.name);
//Start New Recording
dispatch_time_t restartTime = dispatch_time(DISPATCH_TIME_NOW,
0.1 * NSEC_PER_SEC);
dispatch_after(restartTime, dispatch_get_global_queue(0, 0), ^{
AVAudioRecorder *recorder = [[self recorderPool] objectForKey:lastRecID];
NSLog(#"Interruption notification Resumeing recording status %d",recorder.isRecording);
if (recorder) {
if(!recorder.isRecording) {
NSString *filePath = [[self orgFileNames] objectForKey:lastRecID];
NSArray * fileNames =[[self fileNames] objectForKey:lastRecID];
NSString *tmpFileName = [self gnrTempFileName:filePath AndNumber:fileNames.count];
[[[self fileNames] objectForKey:lastRecID] addObject:tmpFileName];
NSURL *url = [NSURL fileURLWithPath:tmpFileName];
NSError *error = nil;
recorder = [[AVAudioRecorder alloc] initWithURL:url settings:recordSetting error:&error];
if (![recorder record]) {
NSLog(#"Interruption notification Error Resumeing recording %#",tempRecorder);
return;
}
[[self recorderPool] setObject:recorder forKey:lastRecID];
NSLog(#"Interruption notification nameResumeing recording %#",lastRecID);
}else {
NSLog(#"Interruption notification Already Recording %d",recorder.isRecording);
}
}else {
NSLog(#"Interruption notification name recording %# not found",lastRecID);
}
});
}
}
}
You will try by using this piece of code
-(IBAction)pauseandplay:(id)sender
{
BOOL status= [player isPlaying];
if(status)
{
[pauseplay setImage:[UIImage imageNamed:#"play.png"]];
[player pause];
}
else
{
[pauseplay setImage:[UIImage imageNamed:#"icon-pause.png"]];
[player play];
updateTimer = [NSTimer scheduledTimerWithTimeInterval:.01 target:self selector:#selector(updateCurrentTime) userInfo:player repeats:YES];
}
}
Here's the problem. I am using AVCaptureVideoDataOutput to get video frames from camera and make video from them with AVAssetWriter. Its working OK, but the video that I get is upside down because default orientation of device for my app is landscape left, not landscape right as its stated by default in AVCaptureVideoDataOutput. Im trying to change orientation in AVCaptureConnection class, but isVideoOrientationSupported is always false, is it somehow possible to fix it?
Here is some code:
AVCaptureDeviceInput *captureInput = [AVCaptureDeviceInput
deviceInputWithDevice:[AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo]
error:nil];
/*We setupt the output*/
AVCaptureVideoDataOutput *captureOutput = [[AVCaptureVideoDataOutput alloc] init];
captureOutput.alwaysDiscardsLateVideoFrames = YES;
captureOutput.minFrameDuration = CMTimeMake(1.0, 24.0); //Uncomment it to specify a minimum duration for each video frame
[captureOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()];
// Set the video output to store frame in BGRA (It is supposed to be faster)
NSString* key = (NSString*)kCVPixelBufferPixelFormatTypeKey;
NSNumber* value = [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA];
NSDictionary* videoSettings = [NSDictionary dictionaryWithObject:value forKey:key];
[captureOutput setVideoSettings:videoSettings];
/*And we create a capture session*/
self.captureSession = [[AVCaptureSession alloc] init];
self.captureSession.sessionPreset = AVCaptureSessionPresetLow;
/*We add input and output*/
if ([self.captureSession canAddInput:captureInput])
{
[self.captureSession addInput:captureInput];
}
if ([self.captureSession canAddOutput:captureOutput])
{
[self.captureSession addOutput:captureOutput];
}
/*We add the preview layer*/
self.prevLayer = [AVCaptureVideoPreviewLayer layerWithSession: self.captureSession];
if ([self.prevLayer isOrientationSupported])
{
[self.prevLayer setOrientation:AVCaptureVideoOrientationLandscapeLeft];
}
self.prevLayer.frame = self.view.bounds;
self.prevLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
[self.view.layer addSublayer: self.prevLayer];
AVCaptureConnection *videoConnection = NULL;
[self.captureSession beginConfiguration];
for ( AVCaptureConnection *connection in [captureOutput connections] )
{
for ( AVCaptureInputPort *port in [connection inputPorts] )
{
if ( [[port mediaType] isEqual:AVMediaTypeVideo] )
{
videoConnection = connection;
}
}
}
if([videoConnection isVideoOrientationSupported]) // **Here it is, its always false**
{
[videoConnection setVideoOrientation:AVCaptureVideoOrientationLandscapeLeft];
}
[self.captureSession commitConfiguration];
[self.captureSession startRunning];
Upd: figured that when exporting video, the AVAssetExportSession loses preferredTransform info.
I ran into the same problem and poked around the AVCamDemo from WWDC. I don't know why (yet) but if you query your videoConnection right after you create all the inputs/outputs/connections then both isVideoOrientationSupported and supportsVideoOrientation return NO.
However, if you query supportsVideoOrientation or isVideoOrientationSupported at some later point (after the GUI is setup for instance) then it will return YES. For instance I query it right after the user clicks the record button just before I call [[self movieFileOutput] startRecordingToOutputFileURL...]
Give it a try, works for me.
From here: http://developer.apple.com/library/ios/#qa/qa1744/_index.html#//apple_ref/doc/uid/DTS40011134
Currently, the capture outputs for a movie file
(AVCaptureMovieFileOutput) and still image (AVCaptureStillImageOutput)
support setting the orientation, but the data output for processing
video frames (AVCaptureVideoDataOutput) does not.
Exposure values from camera can be acquired when you take picture (without saving it to SavedPhotos). A light meter application on iPhone does this, probably by using some private API.
That application does it on iPhone 3GS only, so I guess it may be somehow related to EXIF data which is populated with this information when the image is created.
This all applies to 3GS.
Has anything changed with iPhone OS 4.0?
Is there a regular way to get these values now?
Does anyone have a working code example for taking these camera/photo setting values?
Thank you
If you want realtime* exposure information, you can capture a video using AVCaptureVideoDataOutput. Each frame CMSampleBuffer is full of interesting data describing the current state of the camera.
*up to 30 fps
With AVFoundation in iOS 4.0 you can mess with exposure, refer specifically to AVCaptureDevice, here is a link AVCaptureDevice ref. Not sure if its exactly what you want but you can look around AVFoundation and probably find some useful stuff
I think I finally found the lead to the real EXIF data. It'll be a while before I have actual code to post, but I figured this should be publicized in the meantime.
Google captureStillImageAsynchronouslyFromConnection. It's a function of AVCaptureStillImageOutput and following is an excerpt from the documentation (long sought for):
imageDataSampleBuffer -
The data that was captured.
The buffer attachments may contain metadata appropriate to the image data format. For example, a buffer containing JPEG data may carry a kCGImagePropertyExifDictionary as an attachment. See ImageIO/CGImageProperties.h for a list of keys and value types.
For an example of working with AVCaptureStillImageOutput see WWDC 2010 sample code, under AVCam.
Peace,
O.
Here is the complete solution. Dont forget to import appropriate frameworks and headers.
In the exifAttachments var in capturenow method you'll find all data you are looking for.
#import <AVFoundation/AVFoundation.h>
#import <ImageIO/CGImageProperties.h>
AVCaptureStillImageOutput *stillImageOutput;
AVCaptureSession *session;
- (void)viewDidLoad
{
[super viewDidLoad];
[self setupCaptureSession];
// Do any additional setup after loading the view, typically from a nib.
}
-(void)captureNow{
AVCaptureConnection *videoConnection = nil;
for (AVCaptureConnection *connection in stillImageOutput.connections)
{
for (AVCaptureInputPort *port in [connection inputPorts])
{
if ([[port mediaType] isEqual:AVMediaTypeVideo] )
{
videoConnection = connection;
break;
}
}
if (videoConnection) { break; }
}
[stillImageOutput captureStillImageAsynchronouslyFromConnection:videoConnection
completionHandler:^(CMSampleBufferRef imageDataSampleBuffer, NSError *__strong error) {
CFDictionaryRef exifAttachments = CMGetAttachment( imageDataSampleBuffer, kCGImagePropertyExifDictionary, NULL);
if (exifAttachments)
{
// Do something with the attachments.
NSLog(#"attachements: %#", exifAttachments);
}
else
NSLog(#"no attachments");
NSData *imageData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageDataSampleBuffer];
UIImage *image = [[UIImage alloc] initWithData:imageData];
}];
}
// Create and configure a capture session and start it running
- (void)setupCaptureSession
{
NSError *error = nil;
// Create the session
session = [[AVCaptureSession alloc] init];
// Configure the session to produce lower resolution video frames, if your
// processing algorithm can cope. We'll specify medium quality for the
// chosen device.
session.sessionPreset = AVCaptureSessionPreset352x288;
// Find a suitable AVCaptureDevice
AVCaptureDevice *device = [AVCaptureDevice
defaultDeviceWithMediaType:AVMediaTypeVideo];
[device lockForConfiguration:nil];
device.whiteBalanceMode = AVCaptureWhiteBalanceModeLocked;
device.focusMode = AVCaptureFocusModeLocked;
[device unlockForConfiguration];
// Create a device input with the device and add it to the session.
AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:device
error:&error];
if (!input) {
// Handling the error appropriately.
}
[session addInput:input];
stillImageOutput = [AVCaptureStillImageOutput new];
NSDictionary *outputSettings = [[NSDictionary alloc] initWithObjectsAndKeys: AVVideoCodecJPEG, AVVideoCodecKey, nil];
[stillImageOutput setOutputSettings:outputSettings];
if ([session canAddOutput:stillImageOutput])
[session addOutput:stillImageOutput];
// Start the session running to start the flow of data
[session startRunning];
[self captureNow];
}