RANSAC with SVM

[da...@sibi.fr]

Hello,

Can you please provide a simple example on using RANSAC for input varaible selection of SVM.


Thanks,

David

[Ben Hur Bahia do Nascimento]

Hi,

I'm waiting for something like that too.

Best regards

[César]

Hi there,

Please, can I have some reference or some paper that indicates RANSAC can be used for variable selection? The point is that RANSAC is usually done to identify outliers in a data set and to obtain robust estimates of simple models for those data sets. I believe RANSAC can be used as a way to select input variables, but I am not sure if it should, specially with potentially complex models such as SVMs.

If you have any resources at least citing it could be done, I can try to think on an usage example for it.

Best regards,

Cesar

[da...@sibi.fr]

Hi Cesar,

Thank you to take the time to answer to my question.

As you can guess, i am not a "professional" of Machine learning so excuse my question if it was silly.

but the difficulty i have with svm is the input variable selection. from one set to another, it can lead to great results or just acceptable results. so the question is more on: is there a quantitative way through machine learning, to select best variable for svm/knn...

i tried mutual information of X on y, to select X's, but it does not bring a decisive difference.

Best Regards,

Thank for your unbelievable work.

[César]

Hi David,

Your question wasn't silly, I was curious about it. In the case you would like to detect what features might be more important in your problem, consider trying the Probabilistic Coordinate Descent learning algorithm with a small complexity parameter C. It will create linear logistic regression machines using L1-regularization. Then you can guess which features are most important by checking which weight has the highest absolute value.

Now, something much important that has to be done before you can apply your learning algorithm is to normalize your data to z-scores. You can normalize it by using the Accord.Statistics.Tools.ZScores method. This method will subtract the mean of your variables, and divide them by their standard deviation, transforming them into "dimensionless" variables.

Depending on how many features you have, you may also want to consider interactions between your variables. In this case, you can use the explicit expansion of the quadratic kernel to generate all possible feature combinations by you (but you don't need to, we can keep things simple by now and just try it directly to see what happens). In any case, you can generate all feature permutations by calling the Accord.Statistics.Kernels.Quadratic.Transform() method passing your normalized values.

I hope it helps, please let me know if it works for you!

I might also create a sample application demonstrating how this can be done. Thanks for exposing the problem!

Best regards,

Cesar

[César]

Sorry, I linked the wrong documentation file for the probabilistic coordinate descent learning algorithm for generating L1-logistic regression. This is the correct link.

Hope it helps!

Regards,

Cesar

[ben...@tbit.com.br]

Hi César, how are doing?

I want to know if I can use the current RANSAC algorithm implemented in Accord to train a SVM for multiple classes as described in this article: "RANSAC-SVM for large-scale datasets"¹.

And another question is which training algorithm should I use to train a SVM for multiple classes. Currently I'm using SMO+Grid Search on sigma parameter to find a best model.


(¹) http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4761280&tag=1