iPhone AVErrorInvalidVideoComposition error when doing image overlay on video? - iphone

I have combined the 2 code chunks found here into one solid chunk (and verified the process with an Apple Developer Xcode tutorial file). When I run it, however, I get an error. It says:
Error Domain=AVFoundationErrorDomain Code=-11841 "The operation couldn’t be completed. (AVFoundationErrorDomain error -11841.)"
Any idea why it throws an AVErrorInvalidVideoComposition error? Thanks! (I'm new here so please let me know if you need more info.)
NSURL *videoURL = [info valueForKey:UIImagePickerControllerMediaURL];
/// UIImage into CALayer
UIImage *myImage = [UIImage imageNamed:#"Test.png"];
CALayer *aLayer = [CALayer layer];
aLayer.contents = (id)myImage.CGImage;
AVURLAsset* url = [AVURLAsset URLAssetWithURL:videoURL options:nil];
AVMutableComposition *videoComposition = [[AVMutableComposition alloc] init];
NSError *error;
NSFileManager *fileManager = [NSFileManager defaultManager];
AVMutableCompositionTrack *compositionVideoTrack = [videoComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVAssetTrack *clipVideoTrack = [[url tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
[compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, [url duration]) ofTrack:clipVideoTrack atTime:kCMTimeZero error:&error];
AVMutableVideoComposition* videoComp = [[AVMutableVideoComposition alloc] init];
videoComp.renderSize = CGSizeMake(640, 480);
videoComp.frameDuration = CMTimeMake(1, 30);
CALayer *parentLayer = [CALayer layer];
CALayer *videoLayer = [CALayer layer];
parentLayer.frame = CGRectMake(0, 0, videoComp.renderSize.width, videoComp.renderSize.height);
videoLayer.frame = CGRectMake(0, 0, videoComp.renderSize.width, videoComp.renderSize.height);
[parentLayer addSublayer:videoLayer];
[parentLayer addSublayer:aLayer];
videoComp.animationTool = [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:videoLayer inLayer:parentLayer];
/// instruction
AVMutableVideoCompositionInstruction *instruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
instruction.timeRange = CMTimeRangeMake(kCMTimeZero, CMTimeMakeWithSeconds(60, 30) );
AVMutableVideoCompositionLayerInstruction* layerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:clipVideoTrack];
instruction.layerInstructions = [NSArray arrayWithObject:layerInstruction];
videoComp.instructions = [NSArray arrayWithObject: instruction];
/// outputs
NSString *filePath = nil;
filePath = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
filePath = [filePath stringByAppendingPathComponent:#"temp.mov"];
NSLog(#"exporting to: %#", filePath);
if ([fileManager fileExistsAtPath:filePath])
{
BOOL success = [fileManager removeItemAtPath:filePath error:&error];
if (!success) NSLog(#"FM error: %#", [error localizedDescription]);
}
/// exporting
AVAssetExportSession *exporter;
exporter = [[AVAssetExportSession alloc] initWithAsset:videoComposition presetName:AVAssetExportPresetHighestQuality] ;
exporter.videoComposition = videoComp;
exporter.outputURL=[NSURL fileURLWithPath:filePath];
exporter.outputFileType=AVFileTypeQuickTimeMovie;
[exporter exportAsynchronouslyWithCompletionHandler:^(void){
switch (exporter.status) {
case AVAssetExportSessionStatusFailed:
NSLog(#"exporting failed:%#",exporter.error);
break;
case AVAssetExportSessionStatusCompleted:
NSLog(#"exporting completed");
UISaveVideoAtPathToSavedPhotosAlbum(filePath, self, #selector(video:didFinishSavingWithError:contextInfo:), NULL);
break;
case AVAssetExportSessionStatusCancelled:
NSLog(#"export cancelled");
break;
}
}];

One common problem with invalid compositions is time.
If your various assets that you are combining don't have the same length, make sure the composition you create covers the entire segment.
Check to ensure that your AVMutableVideoCompositionInstruction's timeRange is correct.

Related

How to embed text (caption) a video in iOS in realtime?

I would like to embed a text on a video as it records, then save it in the Camera Roll. When this video is played on any other device this text should show
From here: How can I add overlay text on a video, then re-encode it?
Note this is not updated for ARC, but should give you a good idea about the methodology.
This example includes adding images and text, there may be a faster way using iOS 7 APIs/CoreText.
Also: Real time compositing on a video like this would mean decompressing the necessary frames to bitmaps, drawing the text, then recompressing the video. This is an expensive operation and performance may be poor. This is not the kind of thing that lends itself to mobile applications.
Here is the code to add text and image in the Video file.That may help you.
AVURLAsset* videoAsset = [[AVURLAsset alloc]initWithURL:[NSURL fileURLWithPath:videoName] options:nil];
AVMutableComposition* mixComposition = [AVMutableComposition composition];
AVMutableCompositionTrack *compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVAssetTrack *clipVideoTrack = [[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
[compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:clipVideoTrack atTime:kCMTimeZero error:nil];
[compositionVideoTrack setPreferredTransform:[[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] preferredTransform]];
CGSize videoSize = [clipVideoTrack naturalSize];
UIImage *myImage = [UIImage imageNamed:#"29.png"];
CALayer *aLayer = [CALayer layer];
aLayer.contents = (id)myImage.CGImage;
aLayer.frame = CGRectMake(videoSize.width - 65, videoSize.height - 75, 57, 57);
aLayer.opacity = 0.65;
CALayer *parentLayer = [CALayer layer];
CALayer *videoLayer = [CALayer layer];
parentLayer.frame = CGRectMake(0, 0, videoSize.width, videoSize.height);
videoLayer.frame = CGRectMake(0, 0, videoSize.width, videoSize.height);
[parentLayer addSublayer:videoLayer];
[parentLayer addSublayer:aLayer];
CATextLayer *titleLayer = [CATextLayer layer];
titleLayer.string = #"Text goes here";
titleLayer.font = #"Helvetica";
titleLayer.fontSize = videoSize.height / 6;
//?? titleLayer.shadowOpacity = 0.5;
titleLayer.alignmentMode = kCAAlignmentCenter;
titleLayer.bounds = CGRectMake(0, 0, videoSize.width, videoSize.height / 6); //You may need to adjust this for proper display
[parentLayer addSublayer:titleLayer]; //ONLY IF WE ADDED TEXT
AVMutableVideoComposition* videoComp = [[AVMutableVideoComposition videoComposition] retain];
videoComp.renderSize = videoSize;
videoComp.frameDuration = CMTimeMake(1, 30);
videoComp.animationTool = [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:videoLayer inLayer:parentLayer];
AVMutableVideoCompositionInstruction *instruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
instruction.timeRange = CMTimeRangeMake(kCMTimeZero, [mixComposition duration]);
AVAssetTrack *videoTrack = [[mixComposition tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
AVMutableVideoCompositionLayerInstruction* layerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoTrack];
instruction.layerInstructions = [NSArray arrayWithObject:layerInstruction];
videoComp.instructions = [NSArray arrayWithObject: instruction];
AVAssetExportSession *assetExport = [[AVAssetExportSession alloc] initWithAsset:mixComposition presetName:AVAssetExportPresetHighestQuality];//AVAssetExportPresetPassthrough
assetExport.videoComposition = videoComp;
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString* VideoName = [NSString stringWithFormat:#"%#/mynewwatermarkedvideo.mp4",documentsDirectory];
//NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:VideoName];
NSURL *exportUrl = [NSURL fileURLWithPath:VideoName];
if ([[NSFileManager defaultManager] fileExistsAtPath:VideoName])
{
[[NSFileManager defaultManager] removeItemAtPath:VideoName error:nil];
}
assetExport.outputFileType = AVFileTypeQuickTimeMovie;
assetExport.outputURL = exportUrl;
assetExport.shouldOptimizeForNetworkUse = YES;
//[strRecordedFilename setString: exportPath];
[assetExport exportAsynchronouslyWithCompletionHandler:
^(void ) {
[assetExport release];
dispatch_async(dispatch_get_main_queue(), ^{
[self exportDidFinish:assetExport];
});
}
];
[videoAsset release];
}
-(void)exportDidFinish:(AVAssetExportSession*)session
{
NSURL *exportUrl = session.outputURL;
ALAssetsLibrary *library = [[ALAssetsLibrary alloc] init];
if ([library videoAtPathIsCompatibleWithSavedPhotosAlbum:exportUrl])
{
[library writeVideoAtPathToSavedPhotosAlbum:exportUrl completionBlock:^(NSURL *assetURL, NSError *error)
{
dispatch_async(dispatch_get_main_queue(), ^{
if (error) {
UIAlertView *alert = [[UIAlertView alloc] initWithTitle:#"Error" message:#"Video Saving Failed"
delegate:nil cancelButtonTitle:#"OK" otherButtonTitles:nil];
[alert show];
} else {
UIAlertView *alert = [[UIAlertView alloc] initWithTitle:#"Video Saved" message:#"Saved To Photo Album"
delegate:self cancelButtonTitle:#"OK" otherButtonTitles:nil];
[alert show];
}
});
}];
}
NSLog(#"Completed");
UIAlertView *alert = [[UIAlertView alloc] initWithTitle:#"AlertView" message:#"Video is edited successfully." delegate:self cancelButtonTitle:#"OK" otherButtonTitles:nil];
[alert show];
[alert release];
}

iOS: lowering bitrate of MPMediaItem containing an iPod music

I'm making an app which add a theme music to an video.
Some user complains that if their music is in apple lossless format, the video will be too large.
I found that is because the AVMutableComposition I use just put the original music format in to the video I generated.
So is there any way I can lower the bitrate of the music in MPMediaItem, or change the format it is encoded?
This is a code snippet of the code I use to add music to video.
AVMutableComposition* mixComposition = [AVMutableComposition composition];
AVMutableCompositionTrack *compositionAudioTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio
preferredTrackID:kCMPersistentTrackID_Invalid];
[compositionAudioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration)
ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0]
atTime:kCMTimeZero error:nil];
AVMutableCompositionTrack *compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo
preferredTrackID:kCMPersistentTrackID_Invalid];
[compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration)
ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0]
atTime:kCMTimeZero error:nil];
AVAssetExportSession* _assetExport = [[AVAssetExportSession alloc] initWithAsset:mixComposition
presetName:AVAssetExportPresetPassthrough];
NSURL *exportUrl = [NSURL fileURLWithPath:_videoOutputPath];
if ([[NSFileManager defaultManager] fileExistsAtPath:_videoOutputPath]){
[[NSFileManager defaultManager] removeItemAtPath:_videoOutputPath error:nil];
}
_assetExport.outputFileType = #"com.apple.quicktime-movie";
_assetExport.outputURL = exportUrl;
_assetExport.shouldOptimizeForNetworkUse = YES;
[_assetExport exportAsynchronouslyWithCompletionHandler:^(void ) {}
I finally got it, this is the code I use:
static NSString * const kWriterInputIsReadyForMoreData = #"readyForMoreMediaData";
#import <AVFoundation/AVFoundation.h>
#implementation AudioUtil
{
AVAssetReader *_assetReader;
AVAssetWriter *_assetWriter;
AVAssetWriterInput *_assetWriterInput;
AVAssetReaderTrackOutput *_readerOutput;
void (^_callback)(BOOL);
CMSampleBufferRef _sampleBufferToAppend;
}
-(void)downSamplingAudioWithSourceURL:(NSURL *)sourceURL destinationURL:(NSURL *)destURL timeRange:(CMTimeRange)timeRange callBack:(void (^)(BOOL))callback
{
NSError *error = nil;
_callback = callback;
[[NSFileManager defaultManager] removeItemAtURL:destURL error:nil];
//initialize reader
AVURLAsset *inputAsset = [AVURLAsset assetWithURL:sourceURL];
_assetReader = [[AVAssetReader alloc] initWithAsset:inputAsset error:&error];
_assetReader.timeRange = timeRange;
AVAssetTrack* track = [[inputAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
NSMutableDictionary* audioReadSettings = [NSMutableDictionary dictionary];
audioReadSettings[AVFormatIDKey] = #(kAudioFormatLinearPCM);
audioReadSettings[AVNumberOfChannelsKey] = #([QLVideoFormatProvider audioChannelCount]);
_readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track outputSettings:audioReadSettings];
NSAssert([_assetReader canAddOutput:_readerOutput], #"reader can't add output");
[_assetReader addOutput:_readerOutput];
//initialize writer
_assetWriter = [[AVAssetWriter alloc] initWithURL:destURL fileType:[QLVideoFormatProvider audioFileType] error:nil];
NSMutableDictionary *audioOutputSettings = [NSMutableDictionary dictionary];
audioOutputSettings[AVFormatIDKey] = [QLVideoFormatProvider audioFormatKeyForEncoder];
audioOutputSettings[AVNumberOfChannelsKey] = #([QLVideoFormatProvider audioChannelCount]);
audioOutputSettings[AVSampleRateKey] = #([QLVideoFormatProvider audioSampleRate]);
audioOutputSettings[AVEncoderBitRateKey] = #([QLVideoFormatProvider audioBitrate]);
_assetWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeAudio outputSettings:audioOutputSettings];
[_assetWriter addInput:_assetWriterInput];
//start
[_assetWriter startWriting];
[_assetWriter startSessionAtSourceTime:kCMTimeZero];
BOOL canStartReading = [_assetReader startReading];
NSLog(#"can start reading %d",canStartReading);
if (!canStartReading) {
callback(NO);
return;
}
[_assetWriterInput addObserver:self forKeyPath:kWriterInputIsReadyForMoreData options:NSKeyValueObservingOptionOld|NSKeyValueObservingOptionNew context:NULL];
_sampleBufferToAppend = [_readerOutput copyNextSampleBuffer];
[self appendBufferToAppend];
}
-(void)appendBufferToAppend
{
if ([_assetWriterInput isReadyForMoreMediaData]) {
if (_sampleBufferToAppend) {
[_assetWriterInput appendSampleBuffer:_sampleBufferToAppend];
CFRelease(_sampleBufferToAppend);
}
_sampleBufferToAppend = [_readerOutput copyNextSampleBuffer];
if (_sampleBufferToAppend) {
[self appendBufferToAppend];
}
else {
[_assetWriter finishWritingWithCompletionHandler:^(){
if (_callback) {
_callback(_assetWriter.status == AVAssetWriterStatusCompleted);
};
}];
}
}
else {
}
}
-(void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
{
if ([keyPath isEqualToString:kWriterInputIsReadyForMoreData]) {
if ([change[NSKeyValueChangeNewKey] boolValue] == YES) {
[self appendBufferToAppend];
}
}
}

AVErrorInvalidVideoComposition = -11841

I am merging multiple videos and multiple songs and I am bot getting what is wrong in the code because the same code was running absolutely fine yesterday but today I'm getting the following response
AVAssetExportSessionStatus = 4,error = Error Domain=AVFoundationErrorDomain Code=-11841 "The operation couldn’t be completed. (AVFoundationErrorDomain error -11841.)"
I did some research and found that exporting is getting failed due to invalid video composition.Please find out what is wrong with the video composition.
- (void)mergeAllselectedVideos
{
NSArray *pathArray = [DocumentDirectory getUrlFromDocumentDirectoryOfList:self.selectedClipsArray];
AVMutableComposition *mixComposition = [[AVMutableComposition alloc]init];
NSMutableArray *layerinstructions = [[NSMutableArray alloc]init];
CMTime time = kCMTimeZero;
CMTime previousSongDuration = kCMTimeZero;
for (int i = 0 ; i < pathArray.count; i++)
{
//VIDEO TRACK//
time = CMTimeAdd(time, previousSongDuration);
NSURL *url = [NSURL URLWithString:[pathArray objectAtIndex:i]];
AVAsset *avAsset = [AVAsset assetWithURL:url];
AVMutableCompositionTrack *track = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[track insertTimeRange:CMTimeRangeMake(kCMTimeZero, avAsset.duration) ofTrack:[[avAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:time error:nil];
previousSongDuration = avAsset.duration;
}
CMTime audioTime = kCMTimeZero;
for (int i = 0; i < self.selectedSongsArray.count; i++)
{
MPMediaItem * songItem = [self.selectedSongsArray objectAtIndex:i];
NSURL *songURL = [songItem valueForProperty: MPMediaItemPropertyAssetURL];
AVAsset *audioAsset = [AVAsset assetWithURL:songURL];
AVMutableCompositionTrack *AudioTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
CMTimeRange timeRange = CMTimeRangeMake(audioTime, audioAsset.duration);
if(CMTimeGetSeconds(CMTimeAdd(audioTime, audioAsset.duration)) > CMTimeGetSeconds(time))
{
timeRange = CMTimeRangeMake(audioTime, CMTimeSubtract(time,audioTime));
}
[AudioTrack insertTimeRange:timeRange ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:kCMTimeZero error:nil];
audioTime = CMTimeAdd(audioTime, audioAsset.duration);
}
AVMutableVideoCompositionInstruction * MainInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
MainInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, time);
MainInstruction.layerInstructions = layerinstructions;
AVMutableVideoComposition *MainCompositionInst = [AVMutableVideoComposition videoComposition];
MainCompositionInst.instructions = [NSArray arrayWithObject:MainInstruction];
MainCompositionInst.frameDuration = CMTimeMake(1, 30);
MainCompositionInst.renderSize = CGSizeMake(320.0, 480.0);
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
movieName = [CoreDataFunctions getNameForMovieForDate:[CalendarFunctions getCurrentDateString]];
self.moviePlayButton.titleLabel.text = movieName;
NSString *myPathDocs = [documentsDirectory stringByAppendingPathComponent:movieName];
NSURL *url = [NSURL fileURLWithPath:myPathDocs];
AVAssetExportSession *exporter = [[AVAssetExportSession alloc] initWithAsset:mixComposition presetName:AVAssetExportPresetHighestQuality];
exporter.outputURL=url;
exporter.outputFileType = AVFileTypeQuickTimeMovie;
exporter.videoComposition = MainCompositionInst;
exporter.shouldOptimizeForNetworkUse = YES;
[exporter exportAsynchronouslyWithCompletionHandler:^{dispatch_async(dispatch_get_main_queue(), ^{[self exportDidFinish:exporter];});}];
}
- (void)exportDidFinish:(AVAssetExportSession*)session
{
//Printing error
NSLog(#"AVAssetExportSessionStatus = %i,error = %#",session.status,session.error);
}
I found your question while having the same problem. My theory on this issue is that all the properties for the video composition are not set at export time, so it's crapping out. Here's the stanza that I am now using which is now resulting in an error-free export:
AVMutableVideoComposition *videoComposition = [AVMutableVideoComposition videoComposition];
videoComposition.frameDuration = CMTimeMake(1,30);
videoComposition.renderScale = 1.0;
videoComposition.renderSize = CGSizeMake(352.0, 288.0);
instruction.layerInstructions = [NSArray arrayWithObject: layerInstruction];
instruction.timeRange = CMTimeRangeMake(kCMTimeZero, videoAsset.duration);
videoComposition.instructions = [NSArray arrayWithObject: instruction];
In my case, I was missing the timeRange property on the instruction. Check your own properties to ensure they're getting the correct values. Good luck! This stuff is hard.
If you happen to be passing more than one main instruction to the instructions array of AVMutableVideoComposition, make sure that the time ranges do not overlap or it will cause this error.
You need to set opacity for the first LayerInstruction, e.g.:
[FirstlayerInstruction setOpacity:0.0 atTime:firstAsset.duration];

Merge audio and video when the lengths are different

Im making a video merging a movie file from one image (only have one frame) and an audio of several seconds using this tutorial.
In iphone devices the video duration is equivalent of audio duration and i saw the image along all the video.
But when i share to android devices (through whatsapp) and i press play the playback duration is the movie from images duration (one frame). I make a test that if i create a movie file from one image repeated a hundred times (a 10fps, ten seconds), in android devices playblack duration is ten seconds.
I think that android devices only plays the shortest track in the video, but if i modify the timerange in addMutableTrackWithMediaType of video to the audio duration nothing happens.
Any advice?
Thank you for support
I put all the code here:
-(void) writeImagesToMovieAtPath:(NSString *)path withSize:(CGSize) size {
NSMutableArray *m_PictArray = [NSMutableArray arrayWithCapacity:1];
[m_PictArray addObject:[UIImage imageNamed:#"prueba.jpg"]];
NSString *documentsDirectoryPath = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
NSArray *dirContents = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:documentsDirectoryPath error:nil];
for (NSString *tString in dirContents) {
if ([tString isEqualToString:#"essai.mp4"])
{
[[NSFileManager defaultManager]removeItemAtPath:[NSString stringWithFormat:#"%#/%#",documentsDirectoryPath,tString] error:nil];
}
}
NSLog(#"Write Started");
NSError *error = nil;
AVAssetWriter *videoWriter = [[AVAssetWriter alloc] initWithURL:
[NSURL fileURLWithPath:path] fileType:AVFileTypeMPEG4
error:&error];
NSParameterAssert(videoWriter);
NSDictionary *codecSettings = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:128000], AVVideoAverageBitRateKey,
[NSNumber numberWithInt:15],AVVideoMaxKeyFrameIntervalKey,
AVVideoProfileLevelH264Main30, AVVideoProfileLevelKey,
nil];
NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
AVVideoCodecH264, AVVideoCodecKey,
codecSettings,AVVideoCompressionPropertiesKey,
[NSNumber numberWithInt:size.width], AVVideoWidthKey,
[NSNumber numberWithInt:size.height], AVVideoHeightKey,
nil];
AVAssetWriterInput* videoWriterInput = [[AVAssetWriterInput
assetWriterInputWithMediaType:AVMediaTypeVideo
outputSettings:videoSettings] retain];
AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor
assetWriterInputPixelBufferAdaptorWithAssetWriterInput:videoWriterInput
sourcePixelBufferAttributes:nil];
NSParameterAssert(videoWriterInput);
NSParameterAssert([videoWriter canAddInput:videoWriterInput]);
videoWriterInput.expectsMediaDataInRealTime = YES;
[videoWriter addInput:videoWriterInput];
//Start a session:
[videoWriter startWriting];
[videoWriter startSessionAtSourceTime:kCMTimeZero];
//Video encoding
CVPixelBufferRef buffer = NULL;
//convert uiimage to CGImage.
int frameCount = 0;
for(int i = 0; i<[m_PictArray count]; i++)
{
buffer = [self newPixelBufferFromCGImage:[[m_PictArray objectAtIndex:i] CGImage] andSize:size];
BOOL append_ok = NO;
int j = 0;
while (!append_ok && j < 30)
{
if (adaptor.assetWriterInput.readyForMoreMediaData)
{
printf("appending %d attemp %d\n", frameCount, j);
CMTime frameTime = CMTimeMake(frameCount,(int32_t) 10);
/*
Float64 seconds = 1;
int32_t preferredTimeScale = 10;
CMTime frameTime = CMTimeMakeWithSeconds(seconds, preferredTimeScale);
*/
append_ok = [adaptor appendPixelBuffer:buffer withPresentationTime:frameTime];
CVPixelBufferPoolRef bufferPool = adaptor.pixelBufferPool;
NSParameterAssert(bufferPool != NULL);
[NSThread sleepForTimeInterval:0.05];
}
else
{
printf("adaptor not ready %d, %d\n", frameCount, j);
[NSThread sleepForTimeInterval:0.1];
}
j++;
}
if (!append_ok) {
printf("error appending image %d times %d\n", frameCount, j);
}
frameCount++;
CVBufferRelease(buffer);
}
[videoWriterInput markAsFinished];
[videoWriter finishWriting];
[videoWriterInput release];
[videoWriter release];
[m_PictArray removeAllObjects];
NSLog(#"Write Ended");
[self saveVideoToAlbum:path];
}
-(void)CompileFilesToMakeMovie {
NSLog(#"CompileFilesToMakeMovie");
AVMutableComposition* mixComposition = [AVMutableComposition composition];
NSString *documentsDirectoryPath = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
//Audio file in AAC
NSString* audio_inputFileName = #"zApY4o8QY.m4a";
NSString* audio_inputFilePath = [NSString stringWithFormat:#"%#/%#",[[NSBundle mainBundle] resourcePath],audio_inputFileName];
NSURL* audio_inputFileUrl = [NSURL fileURLWithPath:audio_inputFilePath];
NSString* video_inputFileName = #"essai.mp4";
NSString* video_inputFilePath = [NSString stringWithFormat:#"%#/%#",documentsDirectoryPath,video_inputFileName];
NSURL* video_inputFileUrl = [NSURL fileURLWithPath:video_inputFilePath];
NSString* outputFileName = #"outputFile.mov";
NSString* outputFilePath = [NSString stringWithFormat:#"%#/%#",documentsDirectoryPath,outputFileName];
NSURL* outputFileUrl = [NSURL fileURLWithPath:outputFilePath];
if ([[NSFileManager defaultManager] fileExistsAtPath:outputFilePath])
[[NSFileManager defaultManager] removeItemAtPath:outputFilePath error:nil];
CMTime nextClipStartTime = kCMTimeZero;
AVURLAsset* videoAsset = [[AVURLAsset alloc]initWithURL:video_inputFileUrl options:nil];
AVURLAsset* audioAsset = [[AVURLAsset alloc]initWithURL:audio_inputFileUrl options:nil];
//CMTimeRange video_timeRange = CMTimeRangeMake(kCMTimeZero,videoAsset.duration);
CMTimeRange video_timeRange = CMTimeRangeMake(kCMTimeZero,videoAsset.duration);
AVMutableCompositionTrack *a_compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[a_compositionVideoTrack insertTimeRange:video_timeRange ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:nextClipStartTime error:nil];
CMTimeRange audio_timeRange = CMTimeRangeMake(kCMTimeZero, audioAsset.duration);
AVMutableCompositionTrack *b_compositionAudioTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
[b_compositionAudioTrack insertTimeRange:audio_timeRange ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:nextClipStartTime error:nil];
AVAssetExportSession* _assetExport = [[AVAssetExportSession alloc] initWithAsset:mixComposition presetName:AVAssetExportPresetLowQuality];
_assetExport.shouldOptimizeForNetworkUse = YES;
_assetExport.outputFileType = #"com.apple.quicktime-movie";
_assetExport.outputURL = outputFileUrl;
_assetExport.timeRange = CMTimeRangeMake(kCMTimeZero, audioAsset.duration);
[_assetExport exportAsynchronouslyWithCompletionHandler:
^(void ) {
[self saveVideoToAlbum:outputFilePath];
}
];
NSLog(#"CompileFilesToMakeMovie Finish");
}
- (void) saveVideoToAlbum:(NSString*)path {
NSLog(#"saveVideoToAlbum");
if(UIVideoAtPathIsCompatibleWithSavedPhotosAlbum(path)){
UISaveVideoAtPathToSavedPhotosAlbum (path, self, #selector(video:didFinishSavingWithError: contextInfo:), nil);
}
}
-(void) video:(NSString *)videoPath didFinishSavingWithError:(NSError *)error contextInfo:(void *)contextInfo {
if(error)
NSLog(#"Exportado con error: %#", error);
else
NSLog(#"Exportado OK");
}
- (CVPixelBufferRef) newPixelBufferFromCGImage: (CGImageRef)image andSize:(CGSize)frameSize {
CGAffineTransform frameTransform = CGAffineTransformMake(0, 0, 0, 0, 0, 0);
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, frameSize.width,
frameSize.height, kCVPixelFormatType_32ARGB, (CFDictionaryRef) options,
&pxbuffer);
NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
NSParameterAssert(pxdata != NULL);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, frameSize.width,
frameSize.height, 8, 4*frameSize.width, rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
NSParameterAssert(context);
//CGContextConcatCTM(context, frameTransform);
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return (CVPixelBufferRef)pxbuffer;
}
Just fixed!
I create the movie file repeating X times the image, and then in the composition process i scaled to the size of the audioAsset.duration
CMTimeRange video_timeRange = CMTimeRangeMake(kCMTimeZero,videoAsset.duration);
AVMutableCompositionTrack *a_compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[a_compositionVideoTrack insertTimeRange:video_timeRange ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:kCMTimeZero error:nil];
[a_compositionVideoTrack scaleTimeRange:video_timeRange toDuration:audioAsset.duration];
You need to repeat the image once to allow track to be scaled, but if the movie has only 2 frames, in android only plays eight seconds, so i made a video with a image repeated 10 times to allow me exceed the limit of 45 secons in video share of whatsapp
Inside CompileFilesToMakeMovie method, use video_timeRange instead of audio_timeRange wherever its needed...

AVFoundation + AssetWriter: Generate Movie With Images and Audio

I have to export a movie from my iPhone application which contains UIImage from an NSArray and add some audio files in .caf format that have to start at pre-specified times.
Now I have been able to use the AVAssetWriter (after going through many questions and answers on this and other sites) to export the video portion containing the images but cant seem to find a way to add the audio files to complete the movie.
Here is what I have gotten so far
-(void) writeImagesToMovieAtPath:(NSString *) path withSize:(CGSize) size
{
NSLog(#"Write Started");
NSError *error = nil;
AVAssetWriter *videoWriter = [[AVAssetWriter alloc] initWithURL:
[NSURL fileURLWithPath:path] fileType:AVFileTypeQuickTimeMovie
error:&error];
NSParameterAssert(videoWriter);
NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
AVVideoCodecH264, AVVideoCodecKey,
[NSNumber numberWithInt:size.width], AVVideoWidthKey,
[NSNumber numberWithInt:size.height], AVVideoHeightKey,
nil];
AVAssetWriterInput* videoWriterInput = [[AVAssetWriterInput
assetWriterInputWithMediaType:AVMediaTypeVideo
outputSettings:videoSettings] retain];
AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor
assetWriterInputPixelBufferAdaptorWithAssetWriterInput:videoWriterInput
sourcePixelBufferAttributes:nil];
NSParameterAssert(videoWriterInput);
NSParameterAssert([videoWriter canAddInput:videoWriterInput]);
videoWriterInput.expectsMediaDataInRealTime = YES;
[videoWriter addInput:videoWriterInput];
//Start a session:
[videoWriter startWriting];
[videoWriter startSessionAtSourceTime:kCMTimeZero];
CVPixelBufferRef buffer = NULL;
//convert uiimage to CGImage.
int frameCount = 0;
for(UIImage * img in imageArray)
{
buffer = [self pixelBufferFromCGImage:[img CGImage] andSize:size];
BOOL append_ok = NO;
int j = 0;
while (!append_ok && j < 30)
{
if (adaptor.assetWriterInput.readyForMoreMediaData)
{
printf("appending %d attemp %d\n", frameCount, j);
CMTime frameTime = CMTimeMake(frameCount,(int32_t) kRecordingFPS);
append_ok = [adaptor appendPixelBuffer:buffer withPresentationTime:frameTime];
if(buffer)
CVBufferRelease(buffer);
[NSThread sleepForTimeInterval:0.05];
}
else
{
printf("adaptor not ready %d, %d\n", frameCount, j);
[NSThread sleepForTimeInterval:0.1];
}
j++;
}
if (!append_ok) {
printf("error appending image %d times %d\n", frameCount, j);
}
frameCount++;
}
}
//Finish the session:
[videoWriterInput markAsFinished];
[videoWriter finishWriting];
NSLog(#"Write Ended");
}
And now the code for pixelBufferFromCGImage
- (CVPixelBufferRef) pixelBufferFromCGImage: (CGImageRef) image andSize:(CGSize) size
{
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, size.width,
size.height, kCVPixelFormatType_32ARGB, (CFDictionaryRef) options,
&pxbuffer);
NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
NSParameterAssert(pxdata != NULL);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, size.width,
size.height, 8, 4*size.width, rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
NSParameterAssert(context);
CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return pxbuffer;
}
So can you help me out regarding how to add the audio files and how to make buffers for them and the adaptor and input settings etc
If this approach might cause a problem guide me about how to use a AVMutableComposition to use the image array for video export
I ended up exporting the video separately using the above code and added the audio files separately using AVComposition & AVExportSession.
Here is the code
-(void) addAudioToFileAtPath:(NSString *) filePath toPath:(NSString *)outFilePath
{
NSError * error = nil;
AVMutableComposition * composition = [AVMutableComposition composition];
AVURLAsset * videoAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:filePath] options:nil];
AVAssetTrack * videoAssetTrack = [[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
AVMutableCompositionTrack *compositionVideoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo
preferredTrackID: kCMPersistentTrackID_Invalid];
[compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,videoAsset.duration) ofTrack:videoAssetTrack atTime:kCMTimeZero
error:&error];
CMTime audioStartTime = kCMTimeZero;
for (NSDictionary * audioInfo in audioInfoArray)
{
NSString * pathString = [audioInfo objectForKey:audioFilePath];
AVURLAsset * urlAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:pathString] options:nil];
AVAssetTrack * audioAssetTrack = [[urlAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
AVMutableCompositionTrack *compositionAudioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio
preferredTrackID: kCMPersistentTrackID_Invalid];
[compositionAudioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,urlAsset.duration) ofTrack:audioAssetTrack atTime:audioStartTime error:&error];
audioStartTime = CMTimeAdd(audioStartTime, CMTimeMake((int) (([[audioInfo objectForKey:audioDuration] floatValue] * kRecordingFPS) + 0.5), kRecordingFPS));
}
AVAssetExportSession* assetExport = [[AVAssetExportSession alloc] initWithAsset:composition presetName:AVAssetExportPresetMediumQuality];
assetExport.videoComposition = mutableVideoComposition;
assetExport.outputFileType =AVFileTypeQuickTimeMovie;// #"com.apple.quicktime-movie";
assetExport.outputURL = [NSURL fileURLWithPath:outFilePath];
[assetExport exportAsynchronouslyWithCompletionHandler:
^(void ) {
switch (assetExport.status)
{
case AVAssetExportSessionStatusCompleted:
// export complete
NSLog(#"Export Complete");
break;
case AVAssetExportSessionStatusFailed:
NSLog(#"Export Failed");
NSLog(#"ExportSessionError: %#", [assetExport.error localizedDescription]);
// export error (see exportSession.error)
break;
case AVAssetExportSessionStatusCancelled:
NSLog(#"Export Failed");
NSLog(#"ExportSessionError: %#", [assetExport.error localizedDescription]);
// export cancelled
break;
}
}];
}
Can you please replace the "for" loop with a single "audioInfo" dictionary which has all the values which need to be set so that it becomes more copy-paste friendly? :)
If you just want to add a single audio file, the following code should replace the for loop :
NSString * pathString = [self getAudioFilePath];
AVURLAsset * urlAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:pathString] options:nil];
AVAssetTrack * audioAssetTrack = [[urlAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
AVMutableCompositionTrack *compositionAudioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio
preferredTrackID: kCMPersistentTrackID_Invalid];
[compositionAudioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,urlAsset.duration) ofTrack:audioAssetTrack atTime:kCMTimeZero error:&error];