What is the method of cropping and combining video using AVFoundation? I need to crop away x number of pixels from the top (of a video) and fill that potion of the video with a different video. Is this possible?
Yes, it's possible, but tricky. You need to:
create an AVMutableVideoComposition and add two tracks with the two videos you want to combine.
shift the video on top up by the amount you want. You do this by figuring out the appropriate affine transform and build the AVMutableVideoCompositionInstructions with that affine transform applied.
It's all very messy. These slides will help you through it:
http://www.slideshare.net/invalidname/advanced-av-foundation-cocoaconf-aug-11
Unfortunately, no. There's no way to crop or mask video on the fly in AVFoundation. You could translate the video down the render area (effectively 'cropping' the bottom) using AVMutableVideoComposition and AVMutableVideoCompositionLayerInstructions setTransform:atTime:
Related
If I have understand well, 3D 360 photos are created from a panorama photo, so I guess it should be possible to create a 3D photo (non 360) from a normal photo. But how? I did not find anything in Google! Any idea of what should I search??
So far, if nothing available (I don't think so), I'll try to duplicate the same photo in each eye. One of the pictures a little bit moved to the right, and the other one moved a little bit to the left. But I think the distortion algorithm is much more complicated.
Note: I'm also receiving answers here: https://plus.google.com/u/0/115463690952639951338/posts/4KdqFcqUTT9
I am in no way certain of this, but my intuition on how 3D 360 images are created in GoogleVR is this:
As you take a panorama image, it actually takes a series of images. As you turn the phone around, the perspective changes slightly with each image, not only by angle, but also offset (except in the unlikely event you spin the phone around its own axis). When it stitches together the final image, it creates one image for each eye, picking suitable images from the series so that it creates a 3D effect when viewed together. The same "area" of the image for each eye comes from a different source image.
You can't do anything similar with a single image. It's the multitude of images produced, each with a different perspective coming from the turning of the phone, that enables the algorithm to create a 3D image.
2D lacks a dimension hence cannot be converted to 3D just like that, but there are clever ways for example Google Pixel even though doesn't have 2 camera can make it seem like the image is 3D by applying some Machine learning algorithm that create the effect of perspective and depth by selective blurring.
3d photos can't be taken by normal but you can take 360 photos with normal camera ..... There are many apps via which you can do this ..... Also there are many algorithms to do it programmatically
I currently have a video that has been selected out with a mask and saved as a separate video. The surrounded regions of the mask are all black. I want to superimpose this mask video onto another video of the same dimensions, replacing the black pixels with the pixels in the underlying video. Is this sort of thing possible in Matlab?
Any help would be greatly appreciated!
One easy way out could be to extract frames from both the videos and replace all the black pixels of the first video with the corresponding pixel values from the second video.
I have a video that was taken with a GoPro and I would like to get rid of the fisheye distortion. I know I can get rid of the fisheye with the gopro software, but I want to do this using Matlab instead.
I know there's this http://www.mathworks.com/help/vision/ref/undistortimage.html that applies for images, however, how would I apply it for a full video? The number of frames in the video 207 (it's like 5 - 6 second short video).
Thank you very much!
Can't you just sample your video stream at 24fp (using e.g. ffmpeg, see here ), apply your Matlab routine one frame at a time, then rebuild the video stream in Matlab itself?
You can apply undistortImage to each frame of the video. If the video is saved to a file, you can use vision.VideoFileReader to read it one frame at a time, and then you call undistortImage. Then you can write the undistorted frame to a different file using vision.VideoFileWriter, or you can display it using vision.VideoPlayer.
Of course, this is all assuming that you have calibrated your camera beforehand using the Camera Calibrator App.
Is it possible to composite green screen images -- an animated actor against a green background, with a backdrop photo and make a video of that using avassetwriter on the iPhone.
I have an application that creates a sequence of screenshots of an animated character against a green background. I'd like to composite those with a photograph from their library.
Is there some way to composite the two into a video on the iPhone?
Thanks,
Yes, there is. I just added a chroma key filter to my GPUImage framework, which should let you do realtime green screen effects from camera, image, or movie sources. You just need to use a GPUImageChromaKeyBlendFilter, set the color you want to replace in the first image or video source, set the sensitivity threshold, and optionally set the amount of smoothing to use on colors that are not quite matches of your target.
It acts like the other blend filters in the framework, where you supply the video source to filter as the first input to the filter, and the image or video to replace you target color with as the second input.
I haven't yet tuned this particular filter for performance, but you should easily be able to get 30 FPS processing for 640x480 frames on an older iPhone 4 (~15-20 FPS for 720p).
I am building an iphone app that needs to record grayscale video and save it to the camera roll. I'm stumped at how best to approach this.
I am thinking along the following lines:
Use a shader and opengl to transform the video to grayscale
Use AVFoundation (AVAssetWriter with an AVAssetWriterInputPixelBufferAdaptor) to write the video to the file.
My questions are:
Is this the right approach (simplest, best performance)?
If so, what would be the best way to go from opengl output to a CVPixelBufferRef input for the AVAssetWriterInputPixelBufferAdaptor?
If not, what would be a better approach?
Any nudge in the right direction is much appreciated!
In general, I'd agree with this approach. Doing your processing in an OpenGL ES 2.0 shader should be the most performant way of doing video frame alteration like this, but it won't be very simple. Fortunately, you can start from a pre-existing template that already does this.
You can use the sample application I wrote here (and explained here) as a base. I use custom shaders in this example to track colors in an image, but you could easily alter this to convert the video frames to grayscale (I even saw someone do this once). The code for feeding camera video into a texture and processing it could be used verbatim from that sample.
In one of the display options within that application, I render the processed image first to a framebuffer object, then use glReadPixels() to pull the resulting image back into bytes that I can work with on the CPU. You could use this to get the raw image data back after the GPU has processed a frame, then feed those bytes into CVPixelBufferCreateWithBytes() to generate your CVPixelBufferRef for writing to disk.
(Edit: 2/29/2012) As an update to this, I just implemented this kind of video recording in my open source GPUImage framework, so I can comment on the specific performance for the encoding part of this. It turns out that you can capture video from the camera, perform live filtering on it, grab it from OpenGL ES using glReadPixels(), and write that out as live H.264 video in 640x480 frames on an iPhone 4 at 30 FPS (the maximum camera framerate).
There were a few things that I needed to do in order to get this recording speed. You need to make sure that you set your AVAssetWriterInputPixelBufferAdaptor to use kCVPixelFormatType_32BGRA as its color format for input pixel buffers. Then, you'll need to re-render your RGBA scene using a color-swizzling shader to provide BGRA output when using glReadPixels(). Without this color setting, your video recording framerates will drop to 5-8 FPS on an iPhone 4, where with it they are easily hitting 30 FPS. You can look at the GPUImageMovieWriter class source code to see more about how I did this.
Using the GPUImage framework, your above filtering and encoding task can be handled by simply creating a GPUImageVideoCamera, attaching a target of a GPUImageSaturationFilter with the saturation set to 0, and then attaching a GPUImageMovieWriter as a target of that. The framework will handle the OpenGL ES interactions for you. I've done this, and it works well on all iOS devices I've tested.