How to use KNearestNeighbors with SpeededUpRobustFeaturesDetector

[bara...@gmail.com]

i need to use the KNN in accord.net to match SpeededUpRobustFeaturePoints of two images. is the current datastructure of KNN is provisioned for the same. Can some one help me to find out the solution. I am able to achive the same in emgu CV(open CV).

Here is the code
Bitmap model = new Bitmap("image1.png");
Bitmap observered = new Bitmap("image2.png");

SpeededUpRobustFeaturesDetector surf = new SpeededUpRobustFeaturesDetector();
List<SpeededUpRobustFeaturePoint> points = surf.ProcessImage(model);
List<SpeededUpRobustFeaturePoint> points1 = surf.ProcessImage(observered);

now i need to match the points using KNN. Can anyone help me out?

As well as, i planned to use the Ransac to reduce the incorrect matches. Any light on the same?

Thanks
Barath

[César]

Hi Barath,

You can use the Descriptor property of the SpeededUpRobustFeaturePoint class as the feature vectors for k-NN. However, I am not sure if each point should receive its own label, or something like that. How do you achieve it in EmguCV?

Best regards,

Cesar

[bara...@gmail.com]

Thanks Cesar for the information . This is how i achieved the functionality in EMGU CV. The objective to compare the images for similarity

SURFDetector surfCPU = new SURFDetector(500, false);
VectorOfKeyPoint modelKeyPoints;
VectorOfKeyPoint observedKeyPoints;
Matrix<int> indices;

Matrix<byte> mask;
int k = 2;
double uniquenessThreshold = 0.8;

modelKeyPoints = surfCPU.DetectKeyPointsRaw(modelImage, null);
Matrix<float> modelDescriptors = surfCPU.ComputeDescriptorsRaw(modelImage, null, modelKeyPoints);

watch = Stopwatch.StartNew();

// extract features from the observed image
observedKeyPoints = surfCPU.DetectKeyPointsRaw(observedImage, null);
Matrix<float> observedDescriptors = surfCPU.ComputeDescriptorsRaw(observedImage, null, observedKeyPoints);
BruteForceMatcher<float> matcher = new BruteForceMatcher<float>(DistanceType.L2);
matcher.Add(modelDescriptors);

indices = new Matrix<int>(observedDescriptors.Rows, k);
using (Matrix<float> dist = new Matrix<float>(observedDescriptors.Rows, k))
{
matcher.KnnMatch(observedDescriptors, indices, dist, k, null);
mask = new Matrix<byte>(dist.Rows, 1);
mask.SetValue(255);
Features2DToolbox.VoteForUniqueness(dist, uniquenessThreshold, mask);
}

int nonZeroCount = CvInvoke.cvCountNonZero(mask);
if (nonZeroCount >= 4)
{
nonZeroCount = Features2DToolbox.VoteForSizeAndOrientation(modelKeyPoints, observedKeyPoints, indices, mask, 1.5, 20);
if (nonZeroCount >= 4)
homography = Features2DToolbox.GetHomographyMatrixFromMatchedFeatures(modelKeyPoints, observedKeyPoints, indices, mask, 2);
}

Is this possible to achieve the same functionality with ACCORD.NET.

[César]

Hmm... It should have been, but the current version of the K-Nearest Neighbors classifier didn't have a method for retrieving the "strength" or "score" of the classification in order to retrieve the best match for a given point... I have therefore included those in my current development version, and included a K-NN point matcher as well, but unfortunately it will only be available in the next release of the framework.

If you really need this as soon as possible, I could try sending them to the SVN and you could try to build the project from there, or I could send you just the updated binaries.

[bara...@gmail.com]

I have a database with images. The end user will upload the new image to find out the right match. The application is just like google goggle. I planned to use SURF , and achieved the same using emgu CV. I quiet impressed with ACCORD.NET. I like to stick with accord.net to implement the same functionality. Is my way right , or can you suggest me the better approach to go forward . I am new to the computer vision field, so any suggestion will throw a good light on my precedence.

[bara...@gmail.com]

Great Cesar, it would be helpful really to get the binaries. at the same time my objective is the same as i said in the earlier post. Is the way i following is the right way, because SURF is patented.. At the same time, i not able to match the invariant images such as scale, blur differences etc.. with surf.

[César]

I have attached the latest development binaries of the framework, together with a sample application demonstrating the feature matching using SURF. Hope it can be useful until those features are consolidated into a future release.

Regards,

Cesar

[bara...@gmail.com]

Cheers it works great. Thanks cesar. If we get the api for matching percentage it would be reaaly great. (ie) end user can ask i want 75% simailar images etc.., The emgucv also not have this functionality. Currently in emgucv i am calculating the matching percentage by following formula

((number of matching points/number of query image key points) * 100). It works in most situations. but i feel it is not a stable formula to calculate the matching percentage. If we get this intellignece in API.It would be huge boost for many like me.

[César]

I am glad it worked. Are you computing this formula after the RANSAC filters the points, or before? Have you tried using it only with the raw points found before the RANSAC filtering, but which have a score higher than a threshold?

The KNearestNeighborsMatching object has a property, named Threshold, which can be used to filter points based on their dissimilarity only, unlike RANSAC, which filter points based on spatial constraints. Perhaps you could try playing around with this setting.

Best regards,

Cesar

[mihael....@gmail.com]

This is really great discusion, and i think i am on right way to get the solutions that i am looking for.

I wonder is there any implemented method, or any way how to do it, that comparing to shapes i could get percentageof similarity?

while i have coleration points, what is the best way to compare?

[amin...@optimaken.com]

Hi Cesar,

is it possible to get in touch with you?

Regards.

[hhmm...@gmail.com]

Hi every one
I want to ask you if i can use hashing technique with SURF algorithm,i made a program to make face recognition by matching test image with saved image dataset.
i used Accord.net and made bag of features by BOW of this library then I made ID3 decision tree and KNN but the result in both ways were not very good, i am asking if i can use hashing technique to make fast and better result,or this will not be feasible ?

Regards
Hadeel