When I try to upload videos captured from my iPhone in my app, the server performs a conversion from .mov to .mp4 so that it can be played in other platforms. However the problem is that when I shoot the video (in portrait orientation) and it is converted (using ffmpeg) and then played back from the server, it appears to be rotated. Any idea?
FFMPEG changed the default behavior to auto rotate video sources with rotation metadata in 2015. This was released as v2.7.
If your ffmpeg version is v2.7 or newer, but your rotation metadata isn't respected, the problem is likely that you are using custom rotation based on metadata. This will cause the same logic to be applied twice, changing or cancelling out the rotation.
In addition to removing your custom rotation (recommended), there's an option to turn off auto rotation with -noautorotate.
ffmpeg -noautorotate -i input.mp4...
This will also work in some older releases.
For sake of completeness, the reason this is happening is that iPhones only actually capture video in one fixed orientation. The measured orientation is then recorded in Apple-specific metadata.
The effect is that Quicktime Player reads the metadata and rotates the video to the correct orientation during playback, but other software (e.g., VLC) does not and shows it as oriented in the actual codec data.
This is why rotate=90 (or vflip, or transpose, or etc.) will work for some people, but not others. Depending on how the camera is held during recording, the rotation necessary could be 90, 180, or even 270 degrees. Without reading the metadata, you're just guessing at how much rotation is necessary and the change that fixes one video will fail for another.
What you can also do is remove the QuickTime specific metadata when rotate the .mov.
This will make sure that the video is rotated the same way in VLC and QuickTime
ffmpeg -i in.mov -vf "transpose=1" -metadata:s:v:0 rotate=0 out.mov
Here's the documentation on the -metadata option (from http://ffmpeg.org/ffmpeg.html):
-metadata[:metadata_specifier] key=value (output,per-metadata)
Set a metadata key/value pair.
An optional metadata_specifier may be given to set metadata on streams or chapters. See -map_metadata documentation for details.
This option overrides metadata set with -map_metadata. It is also possible to delete metadata by using an empty value.
For example, for setting the title in the output file:
ffmpeg -i in.avi -metadata title="my title" out.flv
To set the language of the first audio stream:
ffmpeg -i INPUT -metadata:s:a:1 language=eng OUTPUT
Depending on which version of ffmpeg you have and how it's compiled, one of the following should work...
ffmpeg -vf "transpose=1" -i input.mov output.mp4
...or...
ffmpeg -vfilters "rotate=90" -i input.mov output.mp4
Use the vflip filter
ffmpeg -i input.mov -vf "vflip" output.mp4
Rotate did not work for me and transpose=1 was rotating 90 degrees
So - I too ran into this issue, and here my $0.02 on it:
1.) some videos DO have Orientation/Rotation metadata, some don't:
MTS (sony AVHCD) or the AVIs I have - DO NOT have an orientation tag.
MOVs and MP4s (ipad/iphone or samsung galaxy note2) DO HAVE it.
you can check the setting via 'exiftool -Rotation file'.
My videos often have 90 or 180 as the rotation.
2.) ffmpeg - regardless of the man-page with the metadata-tag, just doesn't EVER seem to set it in the output file. - the rotation-tag is ALWAYS '0'.
it correctly reports it in the output - but it's never set right to be reported by exiftool. - But hey - at least it's there and always 0.
3.) rotation angles:
if you want rotate +/- 90: transpose=1 for clockwise 90, 2 ccw
now if you need 180 degree - just add this filter TWICE.
remember - it's a filter-chain you specify. :-) - see further down.
4.) rotate then scale:
this is tricky - because you quickly get into MP4 output format violations.
Let's say you have a 1920x1080 MOV.
rotate by 90 gives 1080x1920
then we rescale to -1:720 -> 1080*(720/1920) = 405 horiz
And 405 horizontal is NOT divisable by 2 - ERROR. fix this manually.
FIXING THIS automatically - requires a bit of shell-script work.
5.) scale then rotate:
you could do it this way - but then you end up with 720x1280. yuck.
But the filter-example here would be:
"-vf yadif=1,scale=-1:720,transpose=1"
It's just not what I want - but could work quite OK.
Putting it all together: - NOTE - 'intentionally WRONG Rotation-tag', just to demonstrate - it won't show up AT ALL in the output !
This will take the input - and rotate it by 180 degree, THEN RESCALE IT - resetting the rotation-tag. - typically iphone/ipad2 can create 180deg rotated material.
you just can leave '-metadata Rotation=x' out the line...
/usr/bin/ffmpeg -i input-movie.mov -timestamp 2012-06-23 08:58:10 -map_metadata 0:0 -metadata Rotation=270 -sws_flags lanczos -vcodec libx264 -x264opts me=umh -b 2600k -vf yadif=1,transpose=1,transpose=1,scale=1280:720 -f mp4 -y output-movie.MP4
I have multiple devices - like a settop box, ipad2, note2, and I convert ALL my input-material (regardless whether it's mp4,mov,MTS,AVI) to 720p mp4, and till now ALL the resulting videos play correct (orientation,sound) on every dev.
Hope it helps.
For including into web pages my portrait-format videos from iPhone, I just discovered the following recipe for getting .mp4 files in portrait display.
Step 1: In QuickTime Player, Export your file to 480p (I assume that 720p or 1080p would work as well). You get a .mov file again.
Step 2: Take the new file in QT Player, and export to “iPad, iPhone…”. You get a .m4v file.
Step 3: I’m using Miro Video Converter, but probably any readily-available converter at all will work, to get your .mp4 file.
Works like a (long-winded) charm.
I've filmed the video with Ipad3 and it was oriented upside down, which I suppose is the common situation of all Apple devices at some versions. Besides of it, the 3-minutes long MOV file (1920x1090) took about 500 Mb in size, which made it not available to share easily. I had to convert it to MP4, and analyzing all threads I've found on stackoverflow, here's the final code string for ffmpeg I've used (ffmpeg ver. 2.8.4):
ffmpeg -i IN.MOV -s 960x540 -metadata:s:v rotate="0" -acodec libmp3lame OUT.mp4
I suppose you may just leave '-metadata:s:v rotate="0"' if you don't need the resize and audio codec change. Note that if you resize the video, width and height should fully divide to 4.
Although the topic is old.
Hope this will help some one:
Get ffmpeg latest version : https://www.ffmpeg.org/download.html
The command that worked for me (to flip 180 degrees):
ffmpeg -noautorotate -i input.mp4 -filter:v "rotate=PI" output.mp4
When the degrees are determined by -filter:v "PI/180*degrees"
for example
-filter:v "45*PI/180" for 45 degrees
A nice explanation is here
https://superuser.com/questions/578321/how-to-rotate-a-video-180-with-ffmpeg
Or... to simply change the tag in an existing file:
Read the current rotation
exiftool -Rotation <file>
then, for example:
exiftool -Rotation=180 <file>
to set it to 180
Related
For the playback of VideoPlayer, there has always been a problem that the audio and video will not be synchronized. The audio track will be later than the video. The degree of the lagging is different on different devices. I have tried to use the VP8 compression format. Some device have improved but some others cannot playback or still lag, the biggest disadvantage is that the file will be fat when packing the AssetBundle files.
I wonder is anyone have the same (or similar) problem or have a solution?
BTW, I dont know how to do if the Audio Output Mode of VideoPlayer selects API Only? I can't find any related usage at present.
My environmental information is:
Unity version:
{Unity2018.4.6f1}
PlayerSettings:
{Platform = Android(min API Level= 16), AutoGraphicsAPI = true, MultithreadedRendering = false}
Project Settings:
{Audio.DefaultSpeakerMode = Stereo, Audio.DSPBufferSize = Best latency, Audio.MaxVirtualVoices = 512, Audio.MaxRealVoices = 32}
VideoPlayer:
{Source = VideoClip, SkipOnDrop = true, AudioOutputMode = Direct}
The format of the video file is mp4:
Format: H.264, AAC, 44100Hz, stereo (LR)
Resolution: 1920x1080
Size: 1.1 MB
FrameRate: 30.14
Data transfer rate (BitRate): 1.23 Mbit / sec
I am not sure if this will help you, but there is a similar question from 4 years back.
Unity play video on Android
I can use AVCaptureDevice as background.contents of SCNScene. And it works, but there is one problem with it. I would like to use video format that has resolution 1920x1080, and 60 FPS. But I can clearly see, that different format is used, it is 30 FPS. I am configuring used device before applying it to background, but somehow SCNScene is changing it. SceneView itself works in 60 FPS, but camera preview is a different story. Can I somehow force SCNScene to use video format I choose?
I know I could just add layer with camera preview under SceneView, but I have reasons why this approach is not working correctly, so I need to use this background property od the scene.
Sample project is here: https://www.dropbox.com/s/b820wxlg8voya58/SampleApp.zip?dl=1
In terminal you can clearly see, that after starting SceneView active format for device changes:
Selected format is:
<AVCaptureDeviceFormat: 0x282d58d00 'vide'/'420v' 1280x 720, { 3- 60 fps}, HRSI:4096x2304, fov:58.632, supports vis, max zoom:120.00 (upscales #2.91), AF System:2, ISO:23.0-736.0, SS:0.000013-0.333333>
2018-10-10 14:47:35.009890+0200 SampleApp[6799:1110910] [SceneKit] Error: Could not get pixel buffer (CVPixelBufferRef)
Format after 3 seconds is:
<AVCaptureDeviceFormat: 0x282d58fb0 'vide'/'420v' 1920x1080, { 3- 30 fps}, HRSI:4096x2304, fov:58.632, supports vis, max zoom:16.00 (upscales #1.94), AF System:2, ISO:23.0-736.0, SS:0.000013-0.333333>
I've built an Unity plugin for my UWP app which converts raw h264 packets to RGB data and renders it to a texture. I've used FFMPEG to do this and it works fine.
int framefinished = avcodec_send_packet(m_pCodecCtx, &packet);
framefinished = avcodec_receive_frame(m_pCodecCtx, m_pFrame);
// YUV to RGB conversion and render to texture after this
Now, I'm trying to shift to hardware based decoding using DirectX11 DXVA2.0.
Using this: https://learn.microsoft.com/en-us/windows/desktop/medfound/supporting-direct3d-11-video-decoding-in-media-foundation
I was able to create a decoder(ID3D11VideoDecoder) but I don't know how to supply it the raw H264 packets and get the YUV or NV12 data as output.
(Or if its possible to render the output directly to the texture since I can get the ID3D11Texture2D pointer)
so my question is, How do you send the raw h264 packets to this decoder and get the output from it?
Also, this is for real time operation so I'm trying to achieve minimal latency.
Thanks in advance!
Since you already have it done using FFMPEG, I'd like to suggest to you to use FFMPEG's dx11 hardware decoding directly.
Check the HW decode FFMPEG example for details: github.com/FFmpeg/FFmpeg/blob/master/doc/examples/hw_decode.c
when I use ffmpeg to scale a H264 video. It seems that the video is decoded to the raw graph then scaled then encoded again. But if the speed is very critical,is there a faster way if I specify a “good" ratio like 2:1, as if I want to pick up one pixel in every four?
I know a bit how h264 works, 8*8/4*4 pixels are coded as a group,so it's not easy to pick up 1/4 pixels in its range. But is there a way to merge 4 group into one quickly?
When you use ffmpeg for scaling purpose, there is no way you can avoid re-encoding to any part of the video. For scale operation, ffmpeg works in pipeline fashion as below:
Decoder ----> Scaler -----> Encoder
Scaler does scale operation only after a completely decoded frame is available to it. Since every packet passes through this pipeline, encoder receives video frames in decompressed (YUV format) form only. So, every YUV frame gets re-encoded after scale operation. I guess it clarifies why there is no way to avoid re-encoding.
The scaling ratio do play a role in complexity. I guess scale ratio 2:1 is OK, scale ratio affects the number of taps (filter coefficients) used in scale algorithm. Also, the scaling algorithm that you may choose add another layer of complexity. Least complex scaling algorithm in ffmpeg is "fast_bilinear". But be aware of the video quality trade-off.
Of course, encoding speed is another factor to consider. It seems you know fairly about it. One thing, see if you can make use of HW decoder & encoder that may be available in your system. If HW codec is available, it greatly improves the speed of this entire pipeline. You can try with -hwaccel dxva2 option for ffmpeg
I want to find a method to eliminate the repeating frames from a video. If I consider a video which will repeatedly show the same frame for 5 seconds, I want to include only one frame of that in the video and make it visible for 5 seconds. In here I am looking forward to minimize the file size by eliminating duplicate frames. Is there a method to do this using Matlab?
If your movie is just a series of stills that you wish to show as a slideshow/presentation with a fixed five second delay, then you should be able to used the 'FrameRate' property for the VideoWriter class. Try something like this example:
writerObj = VideoWriter('peaks.mp4','MPEG-4');
writerObj.FrameRate = 0.2; % One frame every 5 seconds
open(writerObj);
Z = peaks;
surf(Z);
for k = 1:4:20
surf(sin(2*pi*k/20)*Z,Z);
writeVideo(writerObj,getframe);
end
close(writerObj);
However, the frame-rate property cannot be varied over the course of your movie, so the more general form of your question is fundamentally an issue of encoder support for variable frame-rate encoding. Most modern encoders (e.g., H.264 implementations) are not designed to explicitly handle this, but rather have heuristics that can detect when content is not changing and efficiently encode the data (particularly if multi-pass encoding is used). Unfortunately, Matlab (I'm assuming that you've been using the VideoWriter class) doesn't really provide great deal of fidelity in this respect. I'm not even sure what inter-frame encoding settings are used for MPEG-4 with H.264 videos.
If the MPEG-4 with H.264 videos produced by VideoWriter are unacceptable, I'd recommend exporting your video in the highest quality possible (or lossless) and then learn to use a full-fledged encoding framework/library (ffmpeg, libav, x264) or application to encode to the quality and size you desire. Apparently Handbrake has support for variable frame-rate encoding, though it's not necessarily designed for what you may want I've not tested it. Or export your individual still frames and use actually video editing software (e.g. iMovie on OS X). There are also likely dedicated applications that can create a movie from a slideshow/presentation (both PowerPoint and Keynote can do this).
Within Matlab, another alternative, is to use a codec that explicitly supports variable frame rates – QuickTime's image-based codecs: Photo JPEG (not to be confused with Motion-JPEG), Photo PNG (a.k.a. Apple PNG), and Photo TIFF (a.k.a. Apple Tiff). You can encode content directly using these codecs with my QTWriter, which is available on Github. Note however, that on OS X 10.9+, QuickTime Player converts lossless variable frame-rate Photo PNG and Photo TIFF movies to lossy fixed frame-rate H.264 (Photo JPEG movies are not converted). See my note towards the bottom of this page for further details and a workaround.