I'm messing with h264 videos loaded with FFmpeg on the iPhone 3GS. The problem is any videos recorded in "Portrait" orientation have a transformation matrix applied to them causing them to display rotated 90 degrees counter-clock.
From what I understand thus far, I just need to modify the transform matrix in the 'tkhd' atom. The problem is I am having trouble accessing or modifying this data. I checked out the FFmpeg implementation for:
static int mov_read_tkhd(MOVContext *c, ByteIOContext *pb, MOVAtom atom)
which clearly shows how the matrix is accessed in avformat but when I try to access the header bytes using the same functions I am not getting any rational values. Even if I were to successfully pull the matrix I'm not sure how to replace it? FFmpeg has functions for retrieving and appending to the track header but nothing for replace it seems?
Any help would be greatly appreciated.
Thanks,
Matt.
To those who cannot get the v filter option to work, after some searching I found an alternative option which works for my build:
ffmpeg -i in.avi -vf "transpose=1" out.avi
this flips the video 90 degrees clockwise. hope this helps all those on ubuntu struggling like I was!! :)
I just had a patch accepted on the FFMPEG git master branch which should help all of us in adjusting for the iPhone's orientation and subsequent transform applied to the .mov file it sends out. FFMPEG will now send back in metadata:
'rotate'=90,180,or 270 depending on how many degrees you need to turn the image clockwise to get the true rotation.
Portrait video recorded in iPhone 3GS is strangely rotated in landscape, and only quicktime plays it correctly, players like VLC etc play the portrait mode videos in landscape because it is encoded that way! Maybe apple have set some bit so that quicktime identifies and sets the orientation properly.
You can use -vfilters option of ffmpeg to rotate the video. It is available in version r21242. You have to patch it on your version of ffmpeg.
configure the build by using --enable-vfilters
and you can use the option:
ffmpeg -vfilters "rotate=90" -i input.mp4 output.mp4
to rotate the input movie.
the movie is recorded directly with the orientation of the hardware camera. when you turn it the camera still records with the same orientation and the same straight write to file.
What determines the orientation of the video is the Transform matrix
The matrix is set dependent on the iphone's orientation. And this is what will determine if you have to rotate the video. And that rotation could be 90 degrees and it could be 180 degrees. It depends on the phone orientation.
After you get an idea of what the matrix is and how it relates to the image you will be able to determine the orientation of the video.
Related
I recording video using MediaRecorder.When using back-camera,it working fine,but when using front camera,the video captured is being flipped/inverse.Means that the item in right,will appear on the left.The camera preview is working fine,just final captured video flipped.
Here is the camera preview looks like
But the final video appear like this(all the item in left hand side,appear on right hand side)
What I tried so far:
I tried to apply the matrix when prepare recorder,but it seems does change anything.
private boolean prepareRecorder(int cameraId){
//# Create a new instance of MediaRecorder
mRecorder = new MediaRecorder();
setCameraDisplayOrientation(this,cameraId,mCamera);
int angle = getVideoOrientationAngle(this,cameraId);
mRecorder.setOrientationHint(angle);
if(cameraId == Camera.CameraInfo.CAMERA_FACING_FRONT){
Matrix matrix = new Matrix();
matrix.preScale(1.0f,-1.0f);
}
//all other code to prepare recorder here
}
I already read for all this question below,but all this seems didnt solve my problem.For information,I using SurfaceView for the camera preview,so this question here doesn't help.
1) Android flip front camera mirror flipped video
2) How to keep android from inverting the image from the front facing camera?
3) Prevent flipping of the front facing camera
So my question is :
1) How to capture a video by front camera which the video not being inverse(exactly the same with camera preview)?
2) How to achieve this when the Camera preview is using SurfaceView but not TextureView ? (cause all the question I mention above,tell about using TextureView)
All possible solution is mostly welcome..Tq
EDIT
I made 2 short video clip to clarify the problem,please download and take a look
1) The video during camera preview of recording
2) The video of the final product of recording
So, if the system camera app produces video similar to your app, you didn't do something wrong. Now it's time to understand what happens to front-facing camera video recording.
The front facing camera is not different from the rear facing camera in the way it captures still pictures or video. There is a difference how the phone displays camera preview on the screen. To make it look more natural to the user, Android (and all other systems) mirrors the preview, so that you can see yourself as if in a mirror.
It is important to understand that this only applies to the way the preview is presented to you. If you pick up any video conferencing app, connect two devices that you hold in two hands, and look at yourself, you will see to your surprise that the two instances of yourself are flipped.
This is not a bug, this is the natural way to present the video to the other party.
See the sketch:
This is how you see the scene:
This is how your peer sees the same scene
Normally, recording of a video is done from the point if view of your peer, as in the second picture. This is the natural setup for, e.g., video conferencing.
But Snapchat and some other social apps choose to store the front-facing video clip as if you record it from the mirror (as if the recorder is in your hand on the first picture). Some people like this feature, others hate it (see https://forums.androidcentral.com/general-help-how/664539-front-camera-pics-mirrored-reversed-only-snapchat.html and https://www.reddit.com/r/nexus6/comments/3846ay/has_anyone_found_a_fix_for_snapchat_flipping)
You cannot use MediaRecorder for that. You can use the lower-level API of MediaCodec to record processed frames. You need to flip each frame 'manually', and this may be a significant performance hit, because normally the MediaRecorder 'connects' the camera to hardware encoder in a very efficient way, without need even to copy the pixels to user memory. This answer shows how you can manipulate the way camera is rendered to texture.
You can achieve this by recording video manually from surface view.
In such case preview and recording will match exactly.
I've been using this library for this purpose:
https://github.com/spaceLenny/recordablesurfaceview
Here is the guide how to use it (not with camera but with OpenGL drawing): https://withintent.uncorkedstudios.com/recording-screen-video-on-android-with-recordablesurfaceview-451c9daa213e
I’m working on a Matlab application that uses a VLC class to control a VLC-instance. One of the features is to set the VLC player to fullscreen. This feature works perfectly fine.
The VLC player is downloaded from Matlab’s File Exchange: https://se.mathworks.com/matlabcentral/fileexchange/56215-vlc (Thanks a lot Léa Strobino)
However, one particular clip insists on resizing the player to a smaller size.
I have done some research and it turns out that this is a common problem in some VLC versions.
Normal workarounds are to uncheck the “adapt interface to video size” (something like that) and to check the “Fullscreen” box.
This ought to make the player open in fullscreen and not resize the screen to video size. The video still resizes the player to a smaller size.
All the specs of the clips are the same: Same file extension (.vob), formats and were made the same way (I did some video trimming and such using ffmpeg – but the same way every time).
I have noticed one difference and that is that this particular video has a lower Data and bitrate (~1000-1500kbps) where as the others are higher (<4000kbps). Also when showing the properties of the clip the frame height and width are blank as opposed to the others that have specific values.
This should however not have an effect of the fullscreen command from Matlab called after loading the video into the playlist. The command has no effect on this video, but does on all other.
It is possible to set the player to fullscreen manually by clicking the window, so it is not caused by some restriction in the video not allowing it to fullscreen.
Why does the video refuse to go in to fullscreen?
Hope somebody is able to help.
Okay so I seem to have solved the problem now. Without being completely sure why - the problem was in the lowered data/framerate.
I tried to add -crf 18 when converting my .mp4 to a .vob file:
ffmpeg -i input.mp4 -vcodec copy -acodec ac3 -crf 18 output.vob
The -crf stands for Constant Rate Factor and is a way to ensure a specific Data rate. The values goes from 0-51 and 18 seems to be the lowest 'sane' value (highest data rate). A good explanation can be found here: https://superuser.com/questions/677576/what-is-crf-used-for-in-ffmpeg
With this higher data rate the video opens up in fullscreen everytime :=)
I want to repeat 1s video to 10 seconds. I used AVMutableComposition and attached the code below.
When I try with a video recorded by apple default camera app, it works as well.
But I need to have specified video so that I made a customized camera. I made 1s video(frame rate : 5fps, H264 codec).
I am getting black frames with this video.
I am not sure whats the problem. Please help.
I solved it by myself so I did not translate target view's coordinator when apply transform to video track so that it doesn't show up properly.
When I convert .mov files from iPhone 4 and .3gp from Android with FFMPEG.
I need to rotate the video 90 degress, but iPhone 4s needs 180 degress...
Is there any way to fix this?
I don't recall how Android handles this, but iPhones record orientation information in the mov file metadata. Depending on how the camera is held while recording, videos could need 0, 90, 180, or even 270 rotation to play upright after conversion. As yet, I don't know of any additions to ffmpeg to parse this metadata, so you have to do it manually using ffprobe -show_streams or something similar, and then set the rotation appropriately.
Update: as kibitzer points out ffmpeg does now detect that the video is rotated, but does not yet provide an automatic rotation fix. Instead, you'll still have to parse out the rotation information and use a combination of the transpose, vflip, and hflip video filters to correctly rotate the video.
Newer ffmpeg returns a rotate: metadata header (when you run ffmpeg -i), which specifies an angle that the video was rotated, so you don't have to parse the rotation matrix yourself and can just rotate the video by using ffmpeg's filters.
For example, if ffmpeg output rotate: 90 then you would run something like:
ffmpeg -y -i rotated_vid.mov -vf "transpose=1" correct_video.avi
I use a program called mediainfo who show the orientation angle and put in the ffmpeg options:
-vf %ROTATE%
ffmpeg in 95% of the cases understand the orientation but there are some cases, in some hardwares...
I have been facing this since so many days but I have not reach to any conclusion.
My problem is : I want to play an mp3 file but not simply by clicking on a play button.
It is this way I want to play it.
*There is a slider that I can drag using finger, I want that the mp3 should play with the frequency with which I am dragging the finger (or speed with which I am dragging my finger, so that it will give an effect of fast forwarding (funny type of voice)) or if I drag slider slowyly the output will be slow *
The problem is the output of the sound is not coming out smooth. its very distorted and disturbed voice.
I want the outuput to be smoother.
Please help. Any suggestions please. Presently I am using AVAudioPlayer and passing the time value based upon slider input to play the file. (It does not seems to be feasible though).
I feel that it is possible using openAL only and no other way. Because using openAL we can modify the frequency of the sound file (pitch)
CAN SOME ONE PLEASE REFER ME A LINK TO openAL implementation for iPhone . I have never played a sound file using openAL
Help!!
You won't be able to do it with AVAudioPlayer, as it does not support pitch operations.
You can load and decode the entire track into memory for playback with OpenAL (which supports pitch), or you can do realtime loading/decoding and pitch changing using Audio Units (MUCH lower level, and more complicated, though).