I am using Matlab for a position tracking application wherein the position is extracted frame by frame from a ~20 minute .avi file. Right now to process a 20 minute video takes ~1 hour. The annoying thing is that the actual algorithmic computations are quite fast. The bottleneck is simply LOADING the .avi frames into Matlab, which we do 20 frames at a time. Here is our pseudocode:
vidobj = VideoReader(vidFile);
frmStep=20; %# of frames to load at a time
for k=1:frmStep:(numFrames-frmStep+1)
f = read(vidobj, [k (k+frmStep-1)]);
%%Do video processing
end
I was wondering whether there was any way to load this faster or do anything about the horribly long computation times....
Over the years I have tried a couple of alternatives to Matlab's native video processing procedures but I never profiled them so I can't tell you anything about the speed up.
The first alternative I've used extensively was mmread. This function uses ffmpeg to do the actual frame grabbing.
Currently I use the VideoCapture class in mexopencv. You will need opencv installed for that to compile. I have also managed to get most of the Matlab bindings in opencv3 to compile (on Mac OSX), which also gives you a VideoCapture class.
Related
I'm working on processing large video files frame-by-frame. The processing for each frame is independent of other frames (when decompressed) and is also computationally intensive, so I figured parallel processing would be a great way to speed up my analysis. While I have taught myself the basics of using parallel loops, I'm having problems combining the specifics of parfor with VideoReader objects. In my mind I imagine the code running like this
video = VideoReader('video.mp4');
parfor ii = 1 : 90000
frame = read(video, ii);
...analysis on frame...
end
However this warns me against using read() because it will be removed in a future version, so the only alternative I know of is to use frameRead(). However frameRead uses the CurrentTime property of the VideoReader object, which increments itself (according to the fps) each time frameRead is called. This works fine for reading frames in a normal loop, but it makes parfor unhappy because each frame relies on increasing the CurrentTime according to the last. Is there a way to access independent frames in a parallel loop using readFrame or otherwise? I've tried to set the CurrentTime value in each loop by using the loop index and the frame rate like this:
video = VideoReader('video.mp4');
fps = video.FrameRate
results = cell(totalFrames, 1);
parfor ii = 1 : 900000
video.CurrentTime = ii/fps;
frame = readFrame(video);
results{ii} = customAnalysisFunction(frame)
end
In this example the parfor is underlined/flagged and the reason is provided in this message:
MATLAB runs loops in parfor functions by dividing the loop iterations into
groups,and then sending them to MATLAB workers where they run in parallel.
For MATLAB to do this in a repeatable, reliable manner, it must be able to
classify all the variables used in the loop. The code uses the indicated
variable in a way that is incompatible with classification
What steps can I take to be able to read video frames in parallel that is compatible?
Should I just use the read function? What are the reasons why I shouldn't? Are there other video tools for Matlab?
One solution that is often suggested to me is, why don't you split up the video into separate clips? I don't want to do this because it is very slow and requires a lot of extra steps and file handling. It find it hard to believe that there isn't a solution to this problem within Matlab, so I'm looking forward to your answers!
I don't expect that reading frames in parallel would work in MATLAB. The video reader is an object that has an internal state about where it is positioned. You might try to work with a copy of the object. Take a look at this:
http://mathworks.com/help/matlab/ref/matlab.mixin.copyable-class.html
Is there a way to read .avi video file frame by frame using mmread or other function, similar to using videoReader, and readFrame functions?
I used mmread but it took a long time to read each frame as it read all the frames before the specified frame.
In general the media and movie reading in Matlab is cumbersome. I changed to python because of it. I would recommend to split the movie into single image files using tools like avconv or ffmpg and then work on the images, as the image manipulation toolbox is much faster.
If you want to go forward and backward through any movie, especially backwards is very slow. Very often it is implemented, by rewinding and than seeking out your frame, so basically to go from frame 100 to 99, Matlab rewinds to 0 and then seeks through the binary stuff to 99.
I've been trying to to read an MP4 file using VideoReader. Matlab is able to read the images, but the further the frame is along the video, the more time it takes.
tic;I=read(v,1);toc
Elapsed time is 0.264011 seconds.
tic;I=read(v,2000);toc
Elapsed time is 32.859614 seconds.
Also, I'm not sure if this is related, but Matlab cannot determine the number of frames in the file:
v=VideoReader('S1140007 (~200 cubes, large).MP4');
Warning: Unable to determine the number of frames in this file.
I've tried using two versions R2012b and R2015a, and the problem persists.
On a different machine, however, the number of frames can be determined and the reading times don't get longer, so obviously there's something configured wrong on my machine.
I there a known solution for this problem (can this be related to codecs somehow?), or maybe an alternative method of reading one image at a time (readFrame is not relevant for my needs).
Any help would be appreciated,
Aviram
OK, so this is not exactly an answer, but a workaround...
It seems that to set the NumberofFrames property in the videoreader object created for a video with an undetermined number of frames, one needs to read the last frame using the following code (as mentioned in the documentation of VideoReader):
v=VideoReader('path.mp4');
l=read(v,inf);
This sets the number of frames in the video, and allows for indexing and quick reading of single frames from the video. However, this only works in matlab r2012b. In 2015a, the NumberofFrames property is set by the read(v,inf) trick, but the reading is still very time-consuming, for some reason.
I'm not sure why this happens, and as I've said, some of the other machines I've checked were able to read my files properly (but some didn't), so this is far from completed. It is not clear why it cannot determine the number of frames, or why there's any variability between computers and why in some versions the last(v, inf) works and in others only partially.
I am trying to read many video files from a database and process them. I am using Matlab and my problem is that when I want to read a 10 minutes long full HD video I should wait so much and my computer stops performing well. I use this command
VideoReader('movie.mp4')
I have seen that it takes 47 seconds to read a 30 seconds long video in the same format. I do not need to load all frames into my memory I just need 11 frames for each step of my process and really got stock here. Any help will be appreciated.
Also here is my output when I run this command
disp(videoObj);
output:
Summary of Multimedia Reader Object for 'movie.mp4'.
Video Parameters: 30.00 frames per second, RGB24 1280x720.
1482 total video frames available.
By the way I am running my code on Matlab R2014a and my OS is ubuntu 14.0.4.
Siavash,
The long loading time is because the entire file is scanned to determine the number of frames. This process is necessary to support frame indexed based access. In R2014b and higher, the frame counting during construction has been disabled. Additionally, you can seek to specific locations in the file using the CurrentTime property and use the hasFrame/readFrame methods for reading to avoid this performance penalty
Dinesh
I'm using version R2015b and I find the same slow processing of mp4 files with the VideoReader and readFrame functions. However, I find that those functions perform much faster on an avi file than an mp4, so I first convert the mp4 to avi using an independent program from https://www.ffmpeg.org. I don't know why there's such a difference in speed between the two...perhaps someone from MATLAB can provide some insight into that question.
I have a video in which the frame rate is not constant over the length of video. How can i get the actual time information from the video using MATLAB?
Use the function mmread() available from the file exchange: (link). More generally, the best way is to be using a Unix system and issue system calls to a better program, like ffmpeg or mencoder, which can give you frame-specific timing information. Doing this with Matlab only would be a real pain. There may be a toolbox function for it, but then your code only works for other people who also have the toolbox...