My question is related to the following AutoHotkey function (you may find it here):
ImageSearch, OutputVarX, OutputVarY, X1, Y1, X2, Y2, ImageFile
According to its purpose, this function searches a region of the screen for an image.
As of the ImageFile input, an optional parameter is allowed:
*n (variation): Specify for n a number between 0 and 255 (inclusive) to indicate the allowed number of shades of variation in either direction for the intensity of the red, green, and blue components of each pixel's color. For example, *2 would allow two shades of variation. This parameter is helpful if the coloring of the image varies slightly or if ImageFile uses a format such as GIF or JPG that does not accurately represent an image on the screen. If you specify 255 shades of variation, all colors will match. The default is 0 shades.
So far, I've always been forced to "calibrate" that parameter making several trials until the chosen image was found: a too high value would return ErrorLevel = 0 all the time, while a too low one would always return ErrorLevel = 1, and of course both of them are not what you would expect from your script.
My question is: what's the most accurate and efficient way to find the "correct" value of *n, that is, the smallest value of *n that makes ImageSearch to not fail during the search with very high confidence (99% or greater)?
Let a possible attempt to find the following images on screen:
Consider the following case: these are not images that appear or disappear: they simply amend their brightness (what attached above is the "dark" version); this means that a too high *n value would of course make the script to always find them, even if their brightness is high.
According to my trials, a value of *n spanning from 100 to 125 is quite good, but I cannot be sure about that if I don't know a way to accurately calibrate that value.
I've tried to make something such as this script
i = 1
Loop
{
ImageSearch, OutputVarX, OutputVarY, 177, 645, 1150, 686, *%i% C:\...\MyImage.png
i := i + 1
;MsgBox, %i%
if (ErrorLevel = 0)
{
MsgBox, %i%
break
}
}
Return
but this does not seem to work.
I made a function that does this about 2 years ago
I called it VariemClick
I made it to find multiple done buildings in a flash game.
Why? Because in the beginning, the image variation I used, found the image in the flash game, but it only worked one time or a few times and then the next time nothing, so I tried to just set up the used variation, only to have it not find the first image, so I needed some way to first, use a low variation and then a little more and so on, but if it got two high, it just found anything, so in with a max variation.
Now the function will start low and then keep setting the allowed variation up and down within the set min/max value given until a set number of tries and then give you a list with info about the images found and the used variation.
Find it here: http://www.autohotkey.com/board/topic/79607-variemclick-findnclick-images-multiple-times-in-flash-games/
Related
AutoHotkey's PixelSearch allows one to search for a pixel of a certain color in an (X1,Y1)..(X2,Y2) rectangle.
But I want to find any pixel that isn't a certain color, i.e. given a background color, I want to detect when any other color comes up in the foreground.
Is there a way to do this using the built-in PixelSearch, or other built-in functions?
If you have some control over the background color, perhaps you can use PixelSearch's variation parameter.
If your background color is black, search for a white pixel with variation 254. PixelSearch should return any pixel that isn't black.
This code searches for any pixel other than BLACK (untested)
White := 0xFFFFFF
PixelSearch OutputVarX, OutputVarY, X1, Y1, X2, Y2, %White%, 254
If your background is midtone gray (0x7F7F7F), try searching for black and white with variation 127. Again, I haven't tested this. Maybe the variation should be 126 or 63. You might have to engage in some trial and error since the behavior of the variation parameter isn't comprehensively documented.
You can use PixelGetColor to look at an individual pixel. The instruction tells you the color of the pixel at location x, y. An If command determines whether the pixel is 0x9D6346 or not.
Using PixelGetColor in a Loop allows you to test a range of pixels, one at a time by incrementing x, y or both.
And yes the proper order is BGR not RGB. A look at the help file confirmed that. :)
Something is happening to change the color. Perhaps you can look for another way to detect the change on the screen. Is the change of color due to another window or message becoming active? If so there are a number of commands you can use to determine what is appearing on the screen.
The windows spy is good at determining the names of pop up messages. Even if they have the same name as an existing window you can determine which is which by the text included or excluded.
A change of the position of a particular window may be enough to detect the change you are looking for.
Perhaps a bit more detail describing what you are trying to detect will help. It would at least, confirm we are in the same ballpark.
I cannot think of a reliable way to solve this with only built-in functionality. However, you could write it yourself with help of the external GDI+-library.
backgroundColor = 0x644E37
pToken := GDIP_StartUp()
pBitMap := GDIP_BitmapFromScreen()
while(a_index <= a_screenWidth) {
w := a_index
while(a_index <= a_screenHeight) {
h := a_index
ARGB := GDIP_GetPixel(pbitmap, w, h)
RGB := ARGBtoRGB(ARGB)
if(RGB != backgroundColor) {
msgbox, % "found pixel at " w ", " h
}
}
GDIP_DisposeImage(pBitMap)
GDIP_Shutdown(pToken)
return
ARGBtoRGB( ARGB ) {
VarSetCapacity( RGB,6,0 )
DllCall( "msvcrt.dll\sprintf", Str,RGB, Str,"%06X", UInt,ARGB<<8 )
Return "0x" RGB
}
untested
Allegedly, this gdip_bitmapFromScreen() is actually faster than pixelSearch.
source
This pixel search worked for me to find any color other than white:
Search for black (0x000000),
if pixel not found
search for red (0xFF0000),
if pixel not found
search for lime (0x00FF00),
if pixel not found
search for blue (0x0000FF),
if pixel not found, the only color is pure white.
I Set all these to have a variation of 254. A variation of 255 finds everything, including white. I used lime instead of green because the official color called green has a green value of 128, while lime has a green value of 255 (red 0, green 255, blue 0). Seems that just searching for black usually finds NOT white, but sometimes you'll need to work your way down the list. This informs me whether or not there is ANY variation of ANY color other than white, including colors so close to white that I can't see them (when on a white background).
You might have to do a little math and/or experimentation for other colors, but I don't see why this method wouldn't work. I used fast RGB mode and window coordinates. Also I found helpful information here:
https://en.wikipedia.org/wiki/Web_colors
As a relative novice I hate to do this. :)
Right out of PixelSearch in the help file.
PixelSearch, Px, Py, 200, 200, 300, 300, 0x9d6346, 3, Fast
if ErrorLevel
MsgBox, That color was not found in the specified region.
else
MsgBox, A color within 3 shades of variation was found at X%Px% Y%Py%.
Understanding of hexadecimal and how the number applies to the color is necessary.
Color is expressed as a hexadecimal number. In this case 9d6346 and is broken down into three parts. I seem to recall 9d is the value for red, 63 for green, and 46 for blue. Those three numbers can range between Hex 00 to Hex FF (Decimal 0 to 255).
Variation is expressed as a Decimal number is the amount of allowable range in the search color
9D range being 9b to 9f. 63 = 61 to 66, 46 = 44 to 69. Depending on whether the search number is taken in or not my range could be out by 1.
If all else fails Google hexadecimal
Similar possibilities with PixelGetColor followed by a variety of If statements
Be prepared for a very wide variation. Some apps intermingle several distinctly different colors to achieve a desired color.
I'm using MATLAB for image processing and I came across a code with the instruction:
imshow(pixel_labels,[]);
when executed it give a binary image.
I have check the manual of the function on Mathworks.com, the most similar used mode is
imshow(I,[low,high]);
but they don't say a thing about the case where that array is empty ([])
I tried to remove it:
imshow(pixel_labels);
but all I see is a white board. I would like to know what is happening in the first use case (imshow(pixel_labels,[])), I hope from there I will understand why I get a white board in the last use case.
If I type help imshow in MATLAB, the first paragraph reads:
IMSHOW(I,[LOW HIGH]) displays the grayscale image I, specifying the
display
range for I in [LOW HIGH]. The value LOW (and any value less than LOW)
displays as black, the value HIGH (and any value greater than HIGH) displays
as white. Values in between are displayed as intermediate shades of gray,
using the default number of gray levels. If you use an empty matrix ([]) for
[LOW HIGH], IMSHOW uses [min(I(:)) max(I(:))]; that is, the minimum value in
I is displayed as black, and the maximum value is displayed as white.
so [] is simply shorthand for [min(pixel_labels(:)) max(pixel_labels(:))].
I'm not familiar with Local Binary Pattern (LBP), could anyone help me to know how to extract LBP features from facial images (I need a simple code example)?
While searching, I found this code, but I didn't understand it.
So first of all you need to split the face into a certain amount of
sections.
For each of these sections you then have to loop through the all of
the pixels contained within that section and get their value (grey scale or colour values).
For each pixel check the value of the pixels which border it in (diagonals and up down left and right) and save them
for each of the directions check if the colour value of. if the colour is greater than the original pixels value you can assign that value a 1 and if it is less you can assign it as a 0.
you should get a list of 1's and 0's from the previous steps. put these numbers together and it will be a large binary number, you should be able to convert this to decimal and you will have a number assigned for that pixel. save this number per pixel.
after you have got a decimal number for each pixel within a section you can average all of the values to get an average number for this section.
This may not be the best description of how this works so here is a useful picture which might help you.
There is an extractLBPFeatures function in the R2015b release of the Computer Vision System Toolbox for MATLAB.
This question is related to my previous post Image Processing Algorithm in Matlab in stackoverflow, which I already got the results that I wanted to.
But now I am facing another problem, and getting some artefacts in the process images. In my original images (stack of 600 images) I can't see any artefacts, please see the original image from finger nail:
But in my 10 processed results I can see these lines:
I really don't know where they come from?
Also if they belong to the camera's sensor why can't I see them in my original images? Any idea?
Edit:
I have added the following code suggested by #Jonas. It reduces the artefact, but does not completely remove them.
%averaging of images
im = D{1}(:,:);
for i = 2:100
im = imadd(im,D{i}(:,:));
end
im = im/100;
imshow(im,[]);
for i=1:100
SD{i}(:,:)=imsubtract(D{i}(:,:),im(:,:))
end
#belisarius has asked for more images, so I am going to upload 4 images from my finger with speckle pattern and 4 images from black background size( 1280x1024 ):
And here is the black background:
Your artifacts are in fact present in your original image, although not visible.
Code in Mathematica:
i = Import#"http://i.stack.imgur.com/5hM3u.png"
EntropyFilter[i, 1]
The lines are faint, but you can see them by binarization with a very low level threshold:
Binarize[i, .001]
As for what is causing them, I can only speculate. I would start tracing from the camera output itself. Also, you may post two or three images "as they come straight from the camera" to allow us some experimenting.
The camera you're using is most likely has a CMOS chip. Since they have independent column (and possibly row) amplifiers, which may have slightly different electronic properties, you can get the signal from one column more amplified than from another.
Depending on the camera, these variability in column intensity can be stable. In that case, you're in luck: Take ~100 dark images (tape something over the lens), average them, and then subtract them from each image before running the analysis. This should make the lines disappear. If the lines do not disappear (or if there are additional lines), use the post-processing scheme proposed by Amro to remove the lines after binarization.
EDIT
Here's how you'd do the background subtraction, assuming that you have taken 100 dark images and stored them in a cell array D with 100 elements:
% take the mean; convert to double for safety reasons
meanImg = mean( double( cat(3,D{:}) ), 3);
% then you cans subtract the mean from the original (non-dark-frame) image
correctedImage = rawImage - meanImg; %(maybe you need to re-cast the meanImg first)
Here is an answer that in opinion will remove the lines more gently than the above mentioned methods:
im = imread('image.png'); % Original image
imFiltered = im; % The filtered image will end up here
imChanged = false(size(im));% To document the filter performance
% 1)
% Compute the histgrams for each column in the lower part of the image
% (where the columns are most clear) and compute the mean and std each
% bin in the histogram.
histograms = hist(double(im(501:520,:)),0:255);
colMean = mean(histograms,2);
colStd = std(histograms,0,2);
% 2)
% Now loop though each gray level above zero and...
for grayLevel = 1:255
% Find the columns where the number of 'graylevel' pixels is larger than
% mean_n_graylevel + 3*std_n_graylevel). - That is columns that contains
% statistically 'many' pixel with the current 'graylevel'.
lineColumns = find(histograms(grayLevel+1,:)>colMean(grayLevel+1)+3*colStd(grayLevel+1));
% Now remove all graylevel pixels in lineColumns in the original image
if(~isempty(lineColumns))
for col = lineColumns
imFiltered(:,col) = im(:,col).*uint8(~(im(:,col)==grayLevel));
imChanged(:,col) = im(:,col)==grayLevel;
end
end
end
imshow(imChanged)
figure,imshow(imFiltered)
Here is the image after filtering
And this shows the pixels affected by the filter
You could use some sort of morphological opening to remove the thin vertical lines:
img = imread('image.png');
SE = strel('line',2,0);
img2 = imdilate(imerode(img,SE),SE);
subplot(121), imshow(img)
subplot(122), imshow(img2)
The structuring element used was:
>> SE.getnhood
ans =
1 1 1
Without really digging into your image processing, I can think of two reasons for this to happen:
The processing introduced these artifacts. This is unlikely, but it's an option. Check your algorithm and your code.
This is a side-effect because your processing reduced the dynamic range of the picture, just like quantization. So in fact, these artifacts may have already been in the picture itself prior to the processing, but they couldn't be noticed because their level was very close to the background level.
As for the source of these artifacts, it might even be the camera itself.
This is a VERY interesting question. I used to deal with this type of problem with live IR imagers (video systems). We actually had algorithms built into the cameras to deal with this problem prior to the user ever seeing or getting their hands on the image. Couple questions:
1) are you dealing with RAW images or are you dealing with already pre-processed grayscale (or RGB) images?
2) what is your ultimate goal with these images. Is the goal to simply get rid of the lines regardless of the quality in the rest of the image that results, or is the point to preserve the absolute best image quality. Are you to perform other processing afterwards?
I agree that those lines are most likely in ALL of your images. There are 2 reasons for those lines ever showing up in an image, one would be in a bright scene where OP AMPs for columns get saturated, thus causing whole columns of your image to get the brightest value camera can output. Another reason could be bad OP AMPs or ADCs (Analog to Digital Converters) themselves (Most likely not an ADC as normally there is essentially 1 ADC for th whole sensor, which would make the whole image bad, not your case). The saturation case is actually much more difficult to deal with (and I don't think this is your problem). Note: Too much saturation on a sensor can cause bad pixels and columns to arise in your sensor (which is why they say never to point your camera at the sun). The bad column problem can be dealt with. Above in another answer, someone had you averaging images. While this may be good to find out where the bad columns (or bad single pixels, or the noise matrix of your sensor) are (and you would have to average pointing the camera at black, white, essentially solid colors), it isn't the correct answer to get rid of them. By the way, what I am explaining with the black and white and averaging, and finding bad pixels, etc... is called calibrating your sensor.
OK, so saying you are able to get this calibration data, then you WILL be able to find out which columns are bad, even single pixels.
If you have this data, one way that you could erase the columns out is to:
for each bad column
for each pixel (x, y) on the bad column
pixel(x, y) = Average(pixel(x+1,y),pixel(x+1,y-1),pixel(x+1,y+1),
pixel(x-1,y),pixel(x-1,y-1),pixel(x-1,y+1))
What this essentially does is replace the bad pixel with a new pixel which is the average of the 6 remaining good pixels around it. The above is an over-simplified version of an algorithm. There are certainly cases where a singly bad pixel could be right next the bad column and shouldn't be used for averaging, or two or three bad columns right next to each other. One could imagine that you would calculate the values for a bad column, then consider that column good in order to move on to the next bad column, etc....
Now, the reason I asked about the RAW versus B/W or RGB. If you were processing a RAW, depending on the build of the sensor itself, it could be that only one sub-pixel (if you will) of the bayer filtered image sensor has the bad OP AMP. If you could detect this, then you wouldn't necessarily have to throw out the other good sub-pixel's data. Secondarily, if you are using an RGB sensor, to take a grayscale photo, and you shot it in RAW, then you may be able to calculate your own grayscale pixels. Many sensors when giving back a grayscale image when using an RGB sensor, will simply pass back the Green pixel as the overall pixel. This is due to the fact that it really serves as the luminescence of an image. This is why most image sensors implement 2 green sub-pixels for every r or g sub-pixel. If this is what they are doing (not ALL sensors do this) then you may have better luck getting rid of just the bad channel column, and performing your own grayscale conversion using.
gray = (0.299*r + 0.587*g + 0.114*b)
Apologies for the long winded answer, but I hope this is still informational to someone :-)
Since you can not see the lines in the original image, they are either there with low intensity difference in comparison with original range of image, or added by your processing algorithm.
The shape of the disturbance hints to the first option... (Unless you have an algorithm that processes each row separately.)
It seems like your sensor's columns are not uniform, try taking a picture without the finger (background only) using the same exposure (and other) settings, then subtracting it from the photo of the finger (prior to other processing). (Make sure the background is uniform before taking both images.)
For image scanning purposes, I'd like a pixel (which I can get from a UIImage) to match (for a certain percentage) to a pre-set color.
Say pink. When I scan the image for pixels that are pink, I want a function to return a percentage of how much the RGB value in the pixel looks like my pre-set RGB value. This way I'd like all (well, most) pink pixels to become 'visible' to me, not just exact matches.
Is anyone familiar with such an approach? How would you do something like this?
Thanks in advance.
UPDATE: thank you all for your answers so far. I accepted the answer from Damien Pollet because it helped me further and I came to the conclusion that calculating the vector difference between two RGB colors does it perfectly for me (at this moment). It might need some tweaking over time but for now I use the following (in objective c):
float difference = pow( pow((red1 - red2), 2) + pow((green1 - green2), 2) + pow((blue1 - blue2), 2), 0.5 );
If this difference is below 85, I accept the color as my target color. Since my algorithm needs no precision, I'm ok with this solution :)
UPDATE 2: on my search for more I found the following URL which might be quite (understatement) useful for you if you are looking for something similar.
http://www.sunsetlakesoftware.com/2010/10/22/gpu-accelerated-video-processing-mac-and-ios
I would say just compute the vector difference to your target color, and check that it's norm is less than some threshold. I suspect some color spaces are better than others at this, maybe HSL or L*ab, since they separate the brightness from the color hue itself, and so might represent a small perceptual difference by a smaller color vector...
Also, see this related question
Scientific answer: You should convert both colors to the LAB color space and calculate the euclidian distance there. That value is also called deltaE.
The LAB space was developed (using test persons) for exactly that reaason: so that different color pairs with equal distances in tnis space correspond to equal perceived color differences.
However, it sounds like you are not looking for matching a specific color, but rather a color range (lets say all skin tones). That might require more user input than just a reference color + a deltaE tollerance:
a reference color with 3 tollerances for hue, saturation and brightness
a cloud of refence color samples
...