I'm working on processing large video files frame-by-frame. The processing for each frame is independent of other frames (when decompressed) and is also computationally intensive, so I figured parallel processing would be a great way to speed up my analysis. While I have taught myself the basics of using parallel loops, I'm having problems combining the specifics of parfor with VideoReader objects. In my mind I imagine the code running like this
video = VideoReader('video.mp4');
parfor ii = 1 : 90000
frame = read(video, ii);
...analysis on frame...
end
However this warns me against using read() because it will be removed in a future version, so the only alternative I know of is to use frameRead(). However frameRead uses the CurrentTime property of the VideoReader object, which increments itself (according to the fps) each time frameRead is called. This works fine for reading frames in a normal loop, but it makes parfor unhappy because each frame relies on increasing the CurrentTime according to the last. Is there a way to access independent frames in a parallel loop using readFrame or otherwise? I've tried to set the CurrentTime value in each loop by using the loop index and the frame rate like this:
video = VideoReader('video.mp4');
fps = video.FrameRate
results = cell(totalFrames, 1);
parfor ii = 1 : 900000
video.CurrentTime = ii/fps;
frame = readFrame(video);
results{ii} = customAnalysisFunction(frame)
end
In this example the parfor is underlined/flagged and the reason is provided in this message:
MATLAB runs loops in parfor functions by dividing the loop iterations into
groups,and then sending them to MATLAB workers where they run in parallel.
For MATLAB to do this in a repeatable, reliable manner, it must be able to
classify all the variables used in the loop. The code uses the indicated
variable in a way that is incompatible with classification
What steps can I take to be able to read video frames in parallel that is compatible?
Should I just use the read function? What are the reasons why I shouldn't? Are there other video tools for Matlab?
One solution that is often suggested to me is, why don't you split up the video into separate clips? I don't want to do this because it is very slow and requires a lot of extra steps and file handling. It find it hard to believe that there isn't a solution to this problem within Matlab, so I'm looking forward to your answers!
I don't expect that reading frames in parallel would work in MATLAB. The video reader is an object that has an internal state about where it is positioned. You might try to work with a copy of the object. Take a look at this:
http://mathworks.com/help/matlab/ref/matlab.mixin.copyable-class.html
Related
In my current setup I have a for loop in which I extract different type of data from a SQL database hosted on Amazon EC2. This extraction is done in the function extractData(variableName). After that the data gets parsed and stored as a mat file in parsestoreData(data):
variables = {'A','B','C','D','E'}
for i = 1:length(variables)
data = extractData(variables{i});
parsestoreData(data);
end
I would like to parallelize this extraction and parsing of the data and to speed up the process. I argue that I could do this using a parfor instead of for in the above example.
However, I am worried that the extraction will not be improved as the SQL database will get slowed down when multiple requests are made on the same database.
I am therefore wondering if Matlab can handle this issue in a smart way, in terms of parralelization?
The workers in parallel pool running parfor are basically full MATLAB processes without a UI, and they default to running in "single computational thread" mode. I'm not sure whether parfor will benefit you in this case - the parfor loop simply arranges for the MATLAB workers to execute the iterations of your loop in parallel. You can estimate for yourself how well your problem will parallelise by launching multiple full desktop MATLABs, and set them off running your problem simultaneously. I would run something like this:
maxNumCompThreads(1);
while true
t = tic();
data = extractData(...);
parsestoreData(data);
toc(t)
end
and then check how the times reported by toc vary as the number of MATLAB clients varies. If the times remain constant, you could reasonably expect parfor to give you benefit (because it means the body can be parallelised effectively). If however, the times decrease significantly as you run more MATLAB clients, then it's almost certain that parfor would experience the same (relative) slow-down.
Say I have many (around 1000) large matrices (about 1000 by 1000) and I want to add them together element-wise. The very naive way is using a temp variable and accumulates in a loop. For example,
summ=0;
for ii=1:20
for jj=1:20
summ=summ+ rand(400);
end
end
After searching on the Internet for some while, someone said it's better to do with the help of sum(). For example,
sump=zeros(400,400,400);
count=0;
for ii=1:20
for j=1:20
count=count+1;
sump(:,:,count)=rand(400);
end
end
sum(sump,3);
However, after I tested two ways, the result is
Elapsed time is 0.780819 seconds.
Elapsed time is 1.085279 seconds.
which means the second method is even worse.
So I am just wondering if there any effective way to do addition? Assume that I am working on a computer with very large memory and a GTX 1080 (CUDA might be helpful but I don't know whether it's worthy to do so since communication also takes time.)
Thanks for your time! Any reply will be highly appreciated!.
The fastes way is to not use any loops in matlab at all.
In many cases, the internal functions of matlab all well optimized to use SIMD or other acceleration techniques.
An example for using the build in functionalities to create matrices of the desired size is X = rand(sz1,...,szN).
In your explicit case sum(rand(400,400,400),3) should give you then the fastest result.
I am using Matlab for a position tracking application wherein the position is extracted frame by frame from a ~20 minute .avi file. Right now to process a 20 minute video takes ~1 hour. The annoying thing is that the actual algorithmic computations are quite fast. The bottleneck is simply LOADING the .avi frames into Matlab, which we do 20 frames at a time. Here is our pseudocode:
vidobj = VideoReader(vidFile);
frmStep=20; %# of frames to load at a time
for k=1:frmStep:(numFrames-frmStep+1)
f = read(vidobj, [k (k+frmStep-1)]);
%%Do video processing
end
I was wondering whether there was any way to load this faster or do anything about the horribly long computation times....
Over the years I have tried a couple of alternatives to Matlab's native video processing procedures but I never profiled them so I can't tell you anything about the speed up.
The first alternative I've used extensively was mmread. This function uses ffmpeg to do the actual frame grabbing.
Currently I use the VideoCapture class in mexopencv. You will need opencv installed for that to compile. I have also managed to get most of the Matlab bindings in opencv3 to compile (on Mac OSX), which also gives you a VideoCapture class.
I've done a function playSound(f) which looks like this:
function playSound(f)
fs = 44100;
t = 0:1/fs:1;
y = sin(2.*pi.*f.*t);
sound(y,fs,16);
end
I'm using this function in another function with the following code:
while i <= y
b = bin2dec(bits(i:i+k-1));
freq = 2000-frequencyInterval+b*deltaFreq;
playSound(freq);
pause(1);
i = i + k;
end
What I'm expecting from the last piece of code is getting a succession of sinusoids with different frequencies each lasting one second and one after another. But I've noticed that this is completely unreliable and I'm not getting exactly one second of one sin directly followed by another one.
Is there a more precise way to tell Matlab to play my sounds one by one, each during a certain time (no more no less) and without interruptions (or anything else in the middle)?
Matlab pause does not provide millisecond precision, even when measured by its internal tic-toc timer. For example:
tic;pause(1);toc
Elapsed time is 1.006905 seconds.
tic;pause(0.001);toc
Elapsed time is 0.018834 seconds.
If you can't 'pre-render' the entire sound sequence as suggested in the comments, I'd use a tool such as Psychophysics toolbox that can take care both of timing and audio playback.
Playing sound elegantly requires some low-level access and buffering and what not. Matlab's sound() function does not provide such access, and will not do what you want, unfortunately. However, if you've a recent Matlab, you will find audioplayer does do what you want, that's exactly the role it's designed for. Because of the need to 'play nice' with your sound hardware, you will have to invest a little more time in making it behave.
Start at "doc audioplayer".
I'm given the task of reordering a number of randomly placed video frames into the right order. I've managed to do this by using each frame as a reference once, and find the the two closest frames in terms of structure for that reference frame..presumably that these two closest frames would be the ones behind and after that frame in the video. After finding the two closest frames for each video frame, I would then compute a possible path.
My problem however is when it comes to performance, particularly when scoring. It's very inefficient unfortunately,and run time alone for 72 frames (320x240) is approx 80 seconds on just the scoring. I'm not too familiar with Matlab (or any similar language) but this is what I am doing for scoring right now:
for i =1: n_images,
current_image = Images{1,i};
%obtain score pairs image similarity
for j = 1:n_images,
if i ~= j,
scores(1,j) = ssim(Images{1,j}, current_image);
end
end
[svalues, index] = sort(scores,'descend');
Closest(1,i) = index(1,1);
Closest(2,i) = index(1,2);
%Closest consists of a 2 x n_images matrix, where for each frame index, there are two
%column values, which are the indexes of the closest frames.
end
Could anyone give me some pointers for optimizations, or else suggestions for a better way of scoring?
Edit: Images are normalized and converted to grayscale
Edit #2: I've tried using threads by adding parfor in the scoring loop, which improved performance by around 50%, however the problem is that I need to create an executable, and i'm not sure I'd achieve the same performance..
Never mind, here I'm going through all image pairs, twice (switched parameters), which is not needed.So it is possible to reduce the speed by n-1/2.
If you want efficiency over accuracy (which in my case, it is), finding the score from the correlation of histograms is one possible way.
It took me 55 seconds to process 72 frames with ssim(), while only 1.2 seconds with difference of histograms.