I am currently trying to record footage of a camera and represent it with matlab in a graphic window using the "image" command. The problem I'm facing is the slow redraw of the image and this of course effects my whole script. Here's some quick pseudo code to explain my program:
figure
while(true)
Frame = AcquireImageFromCamera(); % Mex, returns current frame
image(I);
end
AcquireImageFromCamera() is a mex coming from an API for the camera.
Now without displaying the acquired image the script easily grabbs all frames coming from the camera (it records with a limited framerate). But as soon as I display every image for a real-time video stream, it slows down terribly and therefore frames are lost as they are not captured.
Does anyone have an idea how I could split the process of acquiring images and displaying them in order to use multiple cores of the CPU for example? Parallel computing is the first thing that pops into my mind, but the parallel toolbox works entirely different form what I want here...
edit: I'm a student and in my faculty's matlab version all toolboxes are included :)
Running two threads or workers is going to be a bit tricky. Instead of that, can you simply update the screen less often? Something like this:
figure
count = 0;
while(true)
Frame = AcquireImageFromCamera(); % Mex, returns current frame
count = count + 1;
if count == 5
count = 0;
image(I);
end
end
Another thing to try is to call image() just once to set up the plot, then update pixels directly. This should be much faster than calling image() every frame. You do this by getting the image handle and changing the CData property.
h = image(I); % first frame only
set(h, 'CData', newPixels); % other frames update pixels like this
Note that updating pixels like this may then require a call to drawnow to show the change on screen.
I'm not sure how precise your pseudo code is, but creating the image object takes quite a bit of overhead. It is much faster to create it once and then just set the image data.
figure
himg = image(I)
while(true)
Frame = AcquireImageFromCamera(); % Mex, returns current frame
set(himg,'cdata',Frame);
drawnow; %Also make sure the screen is actually updated.
end
Matlab has a video player in Computer vision toolbox, which would be faster than using image().
player = vision.VideoPlayer
while(true)
Frame = AcquireImageFromCamera(); % Mex, returns current frame
step(player, Frame);
end
Related
I'm doing a simple GUI in matlab using guide, where I have some sliders to select a frequency or many which then get plotted and played through the speakers. I'm learning real-time sound right now so I thought it'd be neat to incorporate it into the application. I right now have a button that plays the sinusoidal waves for 10 seconds when I press it, but I want a checkbox that disables the button when checked and immediately starts playing the sound continuously until the checkbutton is unchecked again.
I can't really get my head around how to do this in Matlab since I heard Matlab is inherently single threaded. And how to do it right with the GUI and everything since I suspect GUI creation is something you think you do right, but you miss some kind of detail and it only works in some situations.
What I think would be the easy way would be to have the check of the checkbox set the 'Enable' property of the Play button to false, and start an infinite loop that plays it from the real time system object I create. When I however uncheck the checkbox, it should set a flag or something to false and enable the Play button again. Could someone point me to if this is the right approach? How do I make the application listen to clicks on the check box while simultanously being in an infinite loop that processes and plays sound? The simultanous loop should be sensitive to changes in the sliders determining the frequencies and those frequencies are also stored in the handles variable.
Mockup code:
function playbutton_Callback(hObject, eventdata, handles)
freqs = [handles.freq1, handles.freq2, handles.freq3]
hightensecy = sinewave(handles.hightensecx, freqs); %creates a sinewave consisting of the three frequencies. hightensecx is a time vector of ten seconds with a high sample rate.
sound(hightensecy, handles.highsr);
figure;
plot(handles.hightensecx,hightensecy);
function y = sinewave(x, freqs)
% x in sec
y(:,1:size(freqs,2)) = sin(2*pi*x*freqs);
y=sum(y,2)/size(freqs,2);
end
% --- Other code to make the sliders store their values in handles.freq1, handles.freq2... ---
function continuousbox_Callback(hObject, eventdata, handles)
selected = get(hObject,'Value');
if(selected)
set(handles.playbutton, 'Enable', 'off');
startsound(handles);
else
set(handles.playbutton, 'Enable', 'on');
stopsound();
end
% Plays sound continuously
function startsound(handles)
%%Initialization
SamplesPerFrame = 1024;
Fs = handles.highsr;
Player = dsp.AudioPlayer('SampleRate', Fs);
%%Stream
tic
while(1)
%TODO Read in a new 1024 bytes chunk and make sure it has the same
%characteristics as sinewave(x, freqs)
step(Player, outsound)
end
%Stops the continuously played sound
function stopsound()
%TODO Somehow stop the loop inside startsound(handles)
If it is possible, it would be good if the resulting startsound function plays a sound that sounds exactly like the playbutton callback, only indefinitely. I'm also not sure how to handle phase issues that might occur in the real time loop. Should I use a time counter that tics away and calculates the current 1024 chunk's y-value and in that case, should I just use the time distance from tic, or increase it with 1024/sample rate seconds per time? Thanks in advance.
In my experiment I am displaying lots of different gratings. To save time during the experiment's execution, I'm pre-drawing the corresponding textures and keep them in a buffer. However, this takes about 10-20 seconds depending on how many stimuli the run uses, and how fast the machine is.
I was wondering whether there is a way of storing the pre-drawn textures on the hard drive and just have them read in?
Here is the current code:
tic
stimulus_matrix = [];
for ifrequencies = 1:length(frequencies)
for iphase = 1:length(phases)
for icontrast_manipulation = 1:length(contrast_manipulation)
for icontrast_values = 1:length(contrast_values(1,:))
grating = makeStimulus(contrast_values(icontrast_manipulation,icontrast_values),grating_size_degrees,phases(iphase),frequencies(ifrequencies,1));
stimulus_matrix(ifrequencies,iphase,icontrast_manipulation,icontrast_values) = Screen('MakeTexture', my_window, grating);
Screen('FillOval', stimulus_matrix(ifrequencies,iphase,icontrast_manipulation,icontrast_values), background, CenterRect(ovalRect,gratingRect));
% Display a progress bar during buffering
% code for progress bar removed for clarity
end
end
end
end
you can use
save('mydata.mat',stimulus_matrix);
and when you need the matrix, you can load it.
load mydata.mat;
then use stimulus_matrix as you want.
Isn't stimulus_matrix just an index to the generated texture? These indices are just pointers. So if you close the textures but keep the windows open, I don't think loading the matrix enables you to draw textures again.
Not sure if this is ideal for your situation, but you could try:
screen_array = Screen('GetImage', my_window);
while the textures are drawn. This would give you a 3D array of the screen content at that time (including your texture), which you could then edit down to include only your texture, and/or subsequently export.
I am trying to employ a chroma key algorithm on a live video. I need to take a live webcam input, process it in real time and display it. I already have the chroma key algorithm working on images.
How do I process the webcam input and display it immediately. I have tried using snapshot() and passing the image to the chroma key algorithm but it is too slow even if I increase the rate of snapshots. I want a smooth output.
[ Also, if there is a better alternative than Matlab please let me know. ]
Instead of using getsnapshot() which connects to the camera and disconnects on EVERY single frame (thus slow framerates), try to use videoinput and then preview the connection: http://www.mathworks.de/de/help/imaq/preview.html
This example is made for you:
http://www.mathworks.de/products/imaq/code-examples.html?file=/products/demos/shipping/imaq/demoimaq_LiveHistogram.html
As shown you can even define a callback-handler-function which is called on every newly received frame.
You must set TriggerType to manual, or else getsnapshot() will create (and destroy) a conection to the camera everytime you need a frame. By setting it to manual, you can start the camera once, get the frames and stop the camera when you're done.
Here is an example:
vidobj = videoinput('winvideo', 1, 'RGB24_640x480');
triggerconfig(vidobj, 'manual');
start(vidobj);
while true % Or any stop condition
img = getsnapshot(vidobj);
% Process the frame.
...
imshow(img);
drawnow;
end
the getsnapshot function takes a lot of time executing since (I guess) initializes the webcam every time is called. This is a problem if you want to acquire images with a high framerate.
I trick I casually discovered is to call the preview function, which keeps the webcam handler open making getsnapshot almost instantaneous, but it keeps a small preview window open:
% dummy example
cam = videoinput(...);
preview(cam);
while(1)
img = getsnapshot(cam);
% do stuff
end
Is there a "cleaner" way to speedup getsnapshot? (without preview window opened)
You could use the new "machine vision" toolbox which is specially built for vision applications. See code below:
vid = videoinput('winvideo', 1, 'RGB24_320x240'); %select input device
hvpc = vision.VideoPlayer; %create video player object
src = getselectedsource(vid);
vid.FramesPerTrigger =1;
vid.TriggerRepeat = Inf;
vid.ReturnedColorspace = 'rgb';
src.FrameRate = '30';
start(vid)
%start main loop for image acquisition
for t=1:500
imgO=getdata(vid,1,'uint8'); %get image from camera
hvpc.step(imgO); %see current image in player
end
As you can see, you can acquire the image with getdata. The bottleneck in video applications in Matlab was the preview window, which delayed to code substantially. The new vision.VideoPlayer is a lot faster (i have used this code in real time vision applications in Matlab. When i had written the first version without the vision toolbox, achieving frame rates at about 18 fps and using the new toolbox got to around 70!).
Note: I you need speed in image apps using Matlab, you should really consider using OpenCV libs through mex to get a decent performance in image manipulation.
I have written a code to create an animation (satellite movement around the Earth). When I run it, it works fine. However, when it is modified to be part of a code much more complex present in a Matlab GUI, the results produced changes (mainly because of the bigger number of points to plot). I also have noticed that if I use the OpenGL renderer the movement of the satellite is quicker than when the other renderers (Painters and Zbuffer) are used. I do not know if there are further possibilities to achieve an improvement in the rendering of the satellite movement. I think the key is, perhaps, changing the code that creates the actual position of the satellite (handles.psat) and its trajectory along the time (handles.tray)
handles.tray = zeros(1,Fin);
handles.psat = line('parent',ah4,'XData',Y(1,1), 'YData',Y(1,2),...
'ZData',Y(1,3),'Marker','o', 'MarkerSize',10,'MarkerFaceColor','b');
...
while (k<Fin)
az = az + 0.01745329252;
set(hgrot,'Matrix',makehgtform('zrotate',az));
handles.tray(k) = line([Y(k-1,1) Y(k,1)],[Y(k-1,2) Y(k,2)],...
[Y(k-1,3) Y(k,3)],...
'Color','red','LineWidth',3);
set(handles.psat,'XData',Y(k,1),'YData',Y(k,2),'ZData',Y(k,3));
pause(0.02);
k = k + 1;
if (state == 1)
state = 0;
break;
end
end
...
Did you consider to apply a rotation transform matrix on your data instead of the axis?
I think <Though I haven't checked it> that it can speedup your code.
You've used the typical tricks that I use to speed things up, like precomputing the frames, setting XData and YData rather than replotting, and selecting a renderer. Here are a couple more tips though:
1) One thing I noticed in your description is that different renderers and different complexities changed how fast your animation appeared to be running. This is often undesirable. Consider using the actual interval between frames (i.e. use tic; dt = toc) to calculate how far to advance the animation, rather than relying on pause(0.2) to generate a steady frame rate.
2) If the complexity is such that your frame rate is undesirably low, consider replacing pause(0.02) with drawnow, or at least calculate how long to pause on each frame.
3) Try to narrow down the source of your bottleneck a bit further by measuring how long the various steps take. That will let you optimize the right stage of the operation.