Other Classifier

[Omid]

Hi,

I'm Omid, thanks for your awesome library,

Basically, I found a thread on using other classifiers but I didn't find any final answer for it.  I can see in source code that you have other classifiers, but how can I change your example code to use a logistic regression classifier or SVM ... instead of a linear one.

   val config =  LiblinearConfig(cost = .005)
    classifier = trainClassifier(config, featurizer, rawExamples)

Again, thanks for the library.

[David Hall]

Sorry the radio silence. 

Linear classifiers are actually a large class of classifiers that includes logistic regression and SVMs. Liblinear can do all of them. The default is, in fact, logistic regression. ( LiblinearConfig(cost = .005, solverType = SolverType.L2R_LR) 

SolverType.L2R_L1LOSS_SVC_DUAL will give you an SVM. You could also try SolverType.MCSVM_CS

[omid bakhshandeh]

Awesome, Is there any documentation for them? How about kernel methods? That would be great if you document different works that can be done with your library.

Thanks

--

-------------------------------------------
Omid

[David Hall]

It's really just a thin wrapper around Liblinear: http://liblinear.bwaldvogel.de/

I actually didn't write this part of the code and have never used it. Liblinear does not support kernels.

Nak is currently basically unmaintained. We have some new people interested in maintaining it, but they haven't had a chance to do much with it yet.

-- David

[omid bakhshandeh]

How do you use classifiers then ?

--

-------------------------------------------
Omid

[omid bakhshandeh]

So I used other SolverType but I am getting this exception : 

Exception in thread "main" java.lang.IllegalArgumentException: probability output is only supported for logistic regression

--

-------------------------------------------
Omid

[David Hall]

I basically don't use classifiers, really. I build custom structured NLP algorithms; I don't do much consuming of off-the-shelf machine learning algorithms.

[David Hall]

I'd need more a trace to help you with that. Can you give me the first 10 lines or so of the stack trace?

[Omid]

Sure: Here is my code: (it works with the logistic regression version)

val config =  LiblinearConfig(/*cost = 0.05,eps = 0.0001,*/ solverType = nak.liblinear.SolverType.MCSVM_CS)

    classifier = trainClassifier(config, featurizer, rawExamples)

    val comparisons = for (ex <- readRaw(allExamples).toList) yield
      (ex.label, classifier.predict(ex.features), ex.features)


    // Compute and print out the confusion matrix based on the comparisons
    // obtained above.
    val (goldLabels, predictions, inputs) = comparisons.unzip3
    println(ConfusionMatrix(goldLabels, predictions, inputs))

Her is the exception:

Exception in thread "main" java.lang.IllegalArgumentException: probability output is only supported for

logistic regression
 at nak.liblinear.Linear.predictProbability(Linear.java:324)
 at nak.core.LiblinearClassifier$class.apply(Classifier.scala:113)
 at nak.core.Classifier$$anon$2.apply(Classifier.scala:156)
 at nak.core.Classifier$$anon$2.apply(Classifier.scala:156)
 at nak.core.Classifier$class.evalIndexed(Classifier.scala:35)
 at nak.core.Classifier$$anon$2.evalIndexed(Classifier.scala:156)
 at nak.core.IndexedClassifier$class.evalUnindexed(Classifier.scala:50)
 at nak.core.Classifier$$anon$2.evalUnindexed(Classifier.scala:156)
 at nak.core.FeaturizedClassifier$class.evalRaw(Classifier.scala:65)
 at nak.core.Classifier$$anon$2.evalRaw(Classifier.scala:156)
 at nak.core.FeaturizedClassifier$class.predict(Classifier.scala:73)
 at nak.core.Classifier$$anon$2.predict(Classifier.scala:156)
....

[David Hall]

Yeah, looks like it's just not going to work with SVMs, sorry about that. You can try using the liblinear library directly.

-- David

[omidbak...@gmail.com]

I see, so actually there is no implementation  of Machine Learning algorithms in NAK and NAK is wrapper on other libraries?

Anyways, thanks for all efforts

[David Hall]

No, that's not quite true.

There's an SVM implementation (nak.classify.SVM) and a logistic regression implementation (nak.classify.LogisticClassifier) and a perceptron implementation, but they're not battle-hardened like liblinear is. It's also the case that no one is maintaining the library right now--I have my hands full with Breeze and Epic and my other non-public projects--and so I can't really advise using Nak at the moment. Hopefully that will change in the future, but someone needs to breathe life into the project.

-- David

[omid bakhshandeh]

Thanks for the answers. BTW, how would you compare Epic syntactic parser vs Stanford parser? (I have not read Berkeley syntactic parser's paper) 

--

-------------------------------------------
Omid

[Jason Baldridge]

As David mentions, all the things that are in liblinear are in Nak, but currently only logistic regression is made easily accessible through the scala wrapper.

I'm currently unable to dedicate time to Nak, and would love to see others picking it up and moving it along. An immediate, easy way someone could contribute would be to update it to use v0.9 of Breeze.

--

Jason Baldridge
Associate Professor, Dept. of Linguistics, UT Austin
Co-founder & Chief Scientist, People Pattern
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge

[David Hall]

Um, we've had better accuracy than them for a while, though I think they have a newer model that works better. Obviously theirs is an older system, so it's more battle-hardened. I haven't done any head to head comparisons yet, beyond the standard benchmarks that probably aren't indicative of real world performance.

I of course like our API more, and we have a much more permissive license. I'm working on building a model that is trained on more kinds of text (web, biomedical journals, etc.) which should make it more robust.

-- David