Does the sequence of training data matter in SVM?

[Jingkun Gao]

I find it in matlab, when I random the sequence of data, I will get a different result using quadprog. I am thinking does this mean that the sequence of data will influence the starting point, thus influence the final result. If so, how do we set up "a proper starting point" to acquire a good optimization result?

Thanks,

Jingkun

[Xinghai Hu]

I suffers a similar problem. It is wield that only linear and polynomial kernel in dual form perform badly. I have checked the code for quite a lot of time and cannot find a problem.

[Krikamol Muandet]

Hi Jingkun and Xinghai,

The svm solves quadratic programming, which is convex if the kernel is positive semi-definite. So in principle shuffling the data should not alter the solution. It is possible that the kernel matrix computed from the data may  not satisfy the psd condition. I usually add a small constant to the diagonal of the kernel matrix to make it psd.

Did you do cross validation? Doing cross validation to choose the parameter may also help.

Best,

Krik

--
http://alex.smola.org/teaching/cmu2013-10-701 (course website)
http://www.youtube.com/playlist?list=PLZSO_6-bSqHQmMKwWVvYwKreGu4b4kMU9 (YouTube playlist)
---
You received this message because you are subscribed to the Google Groups "10-701 Spring 2013 CMU" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 10-701-spring-201...@googlegroups.com.
To post to this group, send email to 10-701-spri...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Krikamol Muandet
PhD Student                                                   
Max Planck Institute for Intelligent Systems            
Spemannstrasse 38, 72076 Tübingen, Germany      
Telephone: +49-(0)7071 601 554
http://www.kyb.mpg.de/~krikamol

[Jingkun Gao]

Hi Krik and Xinghai,

Thanks for your reply. I understand that the problem is in the optimization part. As you mentioned, we should add a small constant to make the kernel matrix psd, but how to choose that small constant? We can print out the kernel matrix and see how to make it psd, but this is not applicable actually. So, what exactly should we do? When I use quadprog to optimize, I have the code like this 

options = optimset('Algorithm','active-set','MaxIter',30000);

Alpha = quadprog(H,f,[],[],Aeq,Beq,lb,ub,[],options);

Will the parameters we set in options influence the optimization? How to pick up the best parameters in options(test the accuracy of different algorithm to see which one is stable?)?

Another thing you mentioned is to use cross validation to pick up parameter. What parameter do you mean here? C or other parameters in quadprog?

Sorry fo the long questions and thanks again for your patient reply.

Regards,

Jingkun

On Wednesday, February 20, 2013 3:10:43 AM UTC-5, Krik wrote:

Hi Jingkun and Xinghai,

The svm solves quadratic programming, which is convex if the kernel is positive semi-definite. So in principle shuffling the data should not alter the solution. It is possible that the kernel matrix computed from the data may  not satisfy the psd condition. I usually add a small constant to the diagonal of the kernel matrix to make it psd.

Did you do cross validation? Doing cross validation to choose the parameter may also help.

Best,

Krik

On 20 February 2013 09:01, Xinghai Hu <xingh...@west.cmu.edu> wrote:

I suffers a similar problem. It is wield that only linear and polynomial kernel in dual form perform badly. I have checked the code for quite a lot of time and cannot find a problem.

On Wednesday, February 20, 2013 1:06:00 AM UTC-5, Jingkun Gao wrote:

I find it in matlab, when I random the sequence of data, I will get a different result using quadprog. I am thinking does this mean that the sequence of data will influence the starting point, thus influence the final result. If so, how do we set up "a proper starting point" to acquire a good optimization result?

Thanks,

Jingkun

--
http://alex.smola.org/teaching/cmu2013-10-701 (course website)
http://www.youtube.com/playlist?list=PLZSO_6-bSqHQmMKwWVvYwKreGu4b4kMU9 (YouTube playlist)
---
You received this message because you are subscribed to the Google Groups "10-701 Spring 2013 CMU" group.

To unsubscribe from this group and stop receiving emails from it, send an email to 10-701-spring-2013-cmu+unsub...@googlegroups.com.

To post to this group, send email to 10-701-spri...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.