Void’s Vault

Knowledge source for efficiency.

Use Image Acquisition Toolkit in Matlab

I recently spent a lot of time trying to capture videos from multiple webcams at the same time with matlab. I’ll describe what I have found, plus show you some of my code.

First of all, here’s what I was trying to do [look at the joined picture]. I had 5 high OptiTrack cameras (high accuracy) that are tracking the position of a cheap camera on a rod with the help of the optical markers I put on the rod. The rod also have LEDs. The idea is that I have two rods like the one in the picture that will look at each other through the cameras on the rods. That way, I could mesure with the cheap cameras the estimated position of the moving rod (the other rod don’t move) and, at the same time, get the moving rod’s ground truth using the OptiTrack cameras.

OptiTrack is fully integrated into Matlab (Simulink) so it’s quite easy to use these cameras. But as for the cheaper ones, I had to learn how to use the Image Acquisition Toolkit for Matlab.

First, you will have to remember to check how many memory you have left, that way, you will not fall in the good old “Out of Memory” trap, with tons of Blue Screens Of Death, like I did, of course. Here’s a quick how to :

The “memory” command gives you details about the memory you have. If the value of “Maximum possible array” is way smaller than “Memory available for all arrays”, then maybe it’s time to restart Matlab.

From what I could understand, there are to categories where Matlab use memory : image acquisition, and the java heap memory. The second one is for the manipulation of matrix inside matlab. If you use very big data and very big matrix, you need more java heap memory. You can change this amount of memory by clicking (R2012): Environment -> Preferences -> General -> Java Heap Memory. But if you allow more memory to the java heap, less will be available for the image acquisition.

The “imaqmem” command gives you details about the memory you use for image acquisition. This is also the command to use if you want to change “FrameMemoryLimit”, which will increase the maximum number of frame you can acquire without having the “Out of Memory” errors. Remember that if the Java Heap is too high, whatever the limit you set using this command, you won’t get more memory for frame acquisition.

Second, I watched a video from MathWorks that shows how to use the image acquisition. Here’s a quick summary:

1
2
3
info = imaqhwinfo %will give you info about the toolbox
info = imaqhwinfo('winvideo') %Gives you info about your devices
imaqtool %opens up the graphic tool

What is cool about the GUI is that there is an window translating the commands you click on into matlab code, so you will learn quickly how to do that.

If you want to take a snapshot with a camera :

1
2
3
4
vid= videoinput('winvideo', [ID], [FORMAT]);
preview(vid);
pic=getsnapshot(vid);
imshow(pic);

You can’t use getsnapshot to get a regular feed of frames, because snapshot duration will increase over time. It is also not a very good idea to ask the GUI to record an uncompressed avi video, since memory will be filled up in no time. A good approach is to configure manual triggering, so you get instant frames instead of snapshots, to work with the frame and then to flush it from memory.

Finally, here’s the script I wrote in order to get frames from my two cheap cameras:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
%Set up the first webcam
imaqmem(2000000000); %Add enough frame memory to avoid memory error
vA = videoinput('winvideo', 1, 'RGB24_1600x1200');
vA.ReturnedColorspace = 'grayscale';
vA.FramesPerTrigger = 1;
triggerconfig(vA, 'manual');
vA.TriggerRepeat = Inf;% TriggerRepeat is zero based and is always one less than the number of triggers.
src = getselectedsource(vA);
src.BacklightCompensation = 'off';
src.Brightness = 110;
src.Contrast = 255;
src.ExposureMode = 'manual';
src.Exposure = -6;
src.FocusMode = 'manual';
src.Focus = 0;
src.FrameRate = '5.0000';
src.Gain = 255;
src.Sharpness = 0;
src.WhiteBalanceMode = 'manual';
src.WhiteBalance = 10000;
%Setup the second webcam
vB = videoinput('winvideo', 2, 'RGB24_1600x1200');
vB.ReturnedColorspace = 'grayscale';
vB.FramesPerTrigger = 1;
triggerconfig(vB, 'manual');
vB.TriggerRepeat = Inf;% TriggerRepeat is zero based and is always one less than the number of triggers.
src = getselectedsource(vB);
src.BacklightCompensation = 'off';
src.Brightness = 110;
src.Contrast = 255;
src.ExposureMode = 'manual';
src.Exposure = -6;
src.FocusMode = 'manual';
src.Focus = 0;
src.FrameRate = '5.0000';
src.Gain = 255;
src.Sharpness = 0;
src.WhiteBalanceMode = 'manual';
src.WhiteBalance = 10000;

This function is used to get the center position of the LEDs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
function [markers] = GetMarkersCentersOfMass(im, threshold)
    [r,c] = find(im>10);
    markers = cell(2,0);
    if(numel(r) > 0)
        marker(:,1,1) = [r(1);c(1); double(im(r(1),c(1)))];
        splitCount = 1;

        for i=2:1:numel(r)
            id = splitCount + 1;
            for spot = 1:1:splitCount
                if round(sqrt((r(i)-marker(1,numel(find(marker(1,:,spot))),spot)).^2 + (c(i)-marker(2,numel(find(marker(1,:,spot))),spot)).^2)) < threshold
                    id = spot;
                    break;
                end
            end

            if id > splitCount
                splitCount = splitCount + 1;
                marker(:, 1, id) = [r(i);c(i);double(im(r(i),c(i)))];
            else
                marker(:, numel(find(marker(1,:,id)))+1 , id) = [r(i);c(i);double(im(r(i),c(i)))];
            end
        end

        markers = zeros(2,numel(marker(1,1,:)));
        for id = 1:1:numel(markers(1,:))
            markers(:,id) = [sum(marker(1,:,id).*marker(3,:,id))./sum(marker(3,:,id));
                             sum(marker(2,:,id).*marker(3,:,id))./sum(marker(3,:,id))];
        end

        if numel(markers(1,:)) > 2
            %Keep the two highest spots, that is, the lowest y values
            [temp,i] = sort(markers(1,:),'ascend');
            markers = markers(:,i(1:2));
        end
    end
end

And the main code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
clear;

numberOfFrames = 200;
ma = zeros(2,10,numberOfFrames);
mb = zeros(2,10,numberOfFrames);

CamSetup; %see code below (this is the camera configuration part)
start(vA);
start(vB);
pause(10);

for frame = 1:1:numberOfFrames
    trigger(vA);
    trigger(vB);

    while(vA.FramesAvailable == 0 || vB.FramesAvailable == 0); end
    frameA = getdata(vA);
    frameB = getdata(vB);

    %get markers centers of mass
    AMarkersCOM = GetMarkersCentersOfMass(frameA, 60);
    BMarkersCOM = GetMarkersCentersOfMass(frameB, 60);

    ma(:,1:numel(AMarkersCOM(1,:)),frame) = AMarkersCOM;
    mb(:,1:numel(BMarkersCOM(1,:)),frame) = BMarkersCOM;

    %do some private work... Another article will give more details about this secret part!  

    flushdata(vA);
    flushdata(vB);
    clear frameA;
    clear frameB;
end

stop(vA);
stop(vB);