MultiSearch + LibSVM using python-weka-wrapper

[Sebastián Vanrell]

Hi Peter,

I was able to run a parameter optimization over LibSVM using GridSearch (script and log attached). 

Although optimization was working I saw that is taking a lot of time so I wanted to disable cross validation on gridSearch (as a preliminary experiment).

I don't find how to disable cross validation on gridSearch but as MultiSearch seems similar and has the option I decide to give it a try.

I tweak the gridSearch script that I have to use MultiSearch (script and log attached). I think that I'm almost on the right path but settings of "-search" argument are not parsing correctly.

I tried several ways to assign those parameters but no one seems to be corrrect. How do I have to?

BTW, I'm using weka 3.7.12 and python-weka-wrapper 3.0.0.

Cheers, 

Sebastián

[Peter Reutemann]

> I was able to run a parameter optimization over LibSVM using GridSearch
> (script and log attached).
> Although optimization was working I saw that is taking a lot of time so I
> wanted to disable cross validation on gridSearch (as a preliminary
> experiment).
> I don't find how to disable cross validation on gridSearch but as
> MultiSearch seems similar and has the option I decide to give it a try.
>
> I tweak the gridSearch script that I have to use MultiSearch (script and log
> attached). I think that I'm almost on the right path but settings of
> "-search" argument are not parsing correctly.
> I tried several ways to assign those parameters but no one seems to be
> corrrect. How do I have to?

The -search parameter takes a single string argument that is made up
of the classname of the parameter and its options (nested options,
therefore any double quotes have to be escaped). It should look
something like this (single line):

"weka.core.setupgenerator.MathParameter -property classifier.cost -min
-3.0 -max 3.0 -step 1.0 -base 10.0 -expression \"pow(BASE,I)\""

Also note, that the property is prefixed with "classifier.".

> BTW, I'm using weka 3.7.12 and python-weka-wrapper 3.0.0.

To make life a bit easier, I've added wrapper classes for GridSearch
and MultiSearch (requires multisearch-weka-package to be installed).
Check out the updated "classifiers/parameter_optimization.py" example:
https://github.com/fracpete/python-weka-wrapper-examples/blob/ad82cfb97d6b5a11d19837f1f41be920f2a78e11/src/wekaexamples/classifiers/parameter_optimization.py

No new release yet, so you have to install from source code (either
clone the repo or grab a source archive).

Cheers, Peter
--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cms.waikato.ac.nz/~fracpete/ Ph. +64 (7) 858-5174

[Sebastián Vanrell]

Awesome! Thanks Peter!

I let you some comments below.

​​

2015-04-20 0:58 GMT-03:00 Peter Reutemann <frac...@waikato.ac.nz>:

> I was able to run a parameter optimization over LibSVM using GridSearch
> (script and log attached).
> Although optimization was working I saw that is taking a lot of time so I
> wanted to disable cross validation on gridSearch (as a preliminary
> experiment).
> I don't find how to disable cross validation on gridSearch but as
> MultiSearch seems similar and has the option I decide to give it a try.
>
> I tweak the gridSearch script that I have to use MultiSearch (script and log
> attached). I think that I'm almost on the right path but settings of
> "-search" argument are not parsing correctly.
> I tried several ways to assign those parameters but no one seems to be
> corrrect. How do I have to?

The -search parameter takes a single string argument that is made up
of the classname of the parameter and its options (nested options,
therefore any double quotes have to be escaped). It should look
something like this (single line):

"weka.core.setupgenerator.MathParameter -property classifier.cost -min
-3.0 -max 3.0 -step 1.0 -base 10.0 -expression \"pow(BASE,I)\""

​

Before asking you I have tried a single string

​almost correct ​

but whitout the addition of double quotes

​to ​

the -expresion value.

I just use the following one and it works great (using single quotes to define a string and avoid escaping)

'weka.core.setupgenerator.MathParameter -property classifier.cost -min -3.0 -max 3.0 -step 1.0 -base 10.0 -expression "pow(BASE,I)"'

​

 

Also note, that the property is prefixed with "classifier.".

​Using gridSearch I tried with and without "classifier." and for me only works without it. 

Using multiSearch I have to use the prefix, without it doesn't work.

In both cases the classifier used was a LibSVM (which is already a wrapper.... so I have like an inception of wrappers? :P). 

 

> BTW, I'm using weka 3.7.12 and python-weka-wrapper 3.0.0.

To make life a bit easier, I've added wrapper classes for GridSearch
and MultiSearch (requires multisearch-weka-package to be installed).
Check out the updated "classifiers/parameter_optimization.py" example:
https://github.com/fracpete/python-weka-wrapper-examples/blob/ad82cfb97d6b5a11d19837f1f41be920f2a78e11/src/wekaexamples/classifiers/parameter_optimization.py

No new release yet, so you have to install from source code (either
clone the repo or grab a source archive).

​Great add! I will give it a try. Definitly is much simpler.

​

 

Cheers, Peter 

--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cms.waikato.ac.nz/~fracpete/          Ph. +64 (7) 858-5174

--
You received this message because you are subscribed to the Google Groups "python-weka-wrapper" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-weka-wra...@googlegroups.com.
To post to this group, send email to python-we...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python-weka-wrapper/CAHoQ12K4FfYyLQ0uunizbkjCM6rF-4ti6c1UNJHeCxwE%3DfTDTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

​Cheers,

Sebastián​

[Peter Reutemann]

>> The -search parameter takes a single string argument that is made up
>> of the classname of the parameter and its options (nested options,
>> therefore any double quotes have to be escaped). It should look
>> something like this (single line):
>>
>> "weka.core.setupgenerator.MathParameter -property classifier.cost -min
>> -3.0 -max 3.0 -step 1.0 -base 10.0 -expression \"pow(BASE,I)\""
>
>
> Before asking you I have tried a single string
> almost correct
> but whitout the addition of double quotes
> to
> the -expresion value.
> I just use the following one and it works great (using single quotes to
> define a string and avoid escaping)
> 'weka.core.setupgenerator.MathParameter -property classifier.cost -min -3.0
> -max 3.0 -step 1.0 -base 10.0 -expression "pow(BASE,I)"'

True, forgot about that. However, if you have to nest deeper, then you
have to escape the double quotes with backslashes.

>> Also note, that the property is prefixed with "classifier.".
>
>
> Using gridSearch I tried with and without "classifier." and for me only
> works without it.

I was only referring to MultiSearch, since that was the topic of your
query. The "classifier" property refers to the MultiSearch object's
property "classifier". For GridSearch, you don't specify this prefix.

> Using multiSearch I have to use the prefix, without it doesn't work.

Correct.

> In both cases the classifier used was a LibSVM (which is already a
> wrapper.... so I have like an inception of wrappers? :P).

Not quite. ;-) MultiSearch uses a different approach to setting the
properties, hence the requirement of the "classifier." prefix.

>> > BTW, I'm using weka 3.7.12 and python-weka-wrapper 3.0.0.
>>
>> To make life a bit easier, I've added wrapper classes for GridSearch
>> and MultiSearch (requires multisearch-weka-package to be installed).
>> Check out the updated "classifiers/parameter_optimization.py" example:
>>
>> https://github.com/fracpete/python-weka-wrapper-examples/blob/ad82cfb97d6b5a11d19837f1f41be920f2a78e11/src/wekaexamples/classifiers/parameter_optimization.py
>>
>> No new release yet, so you have to install from source code (either
>> clone the repo or grab a source archive).
>
>
> Great add! I will give it a try. Definitly is much simpler.

:-)