Out-of-the-box pylearn2 supported "Unsupervised Feature Learning" modules

[Tom Szumowski]

Hi everyone, first time poster and pylearn2 beginner here.

I'm going through the code/tutorials to get a feel for how pylearn2 works. I noticed an interesting note in the file pylearn2/scripts/tutorials/deep_trainer/readme.txt:

This is not the best-supported tutorial in pylearn2, in part because most of
the active pylearn2 developers are no longer researching stacked layerwise
pretraining.

Just curious ... what is the active research area for the pylearn2 developers for unsupervised feature learning right now?

Thanks!

-Tom

[Ian Goodfellow]

Most of us don't do unsupervised feature learning anymore. 

--
You received this message because you are subscribed to the Google Groups "pylearn-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylearn-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sent from Gmail Mobile

[Tom Szumowski]

Oh ok. Thank you for the quick response. Is that because more recent techniques have made the old ways of "pre-training" no longer necessary? Or have the devs just simply moved to a different research area?

-Tom

On Thursday, February 20, 2014 9:54:30 PM UTC-5, Ian Goodfellow wrote:

Most of us don't do unsupervised feature learning anymore. 

Il giovedì 20 febbraio 2014, Tom Szumowski <tszum...@gmail.com> ha scritto:

Hi everyone, first time poster and pylearn2 beginner here.

I'm going through the code/tutorials to get a feel for how pylearn2 works. I noticed an interesting note in the file pylearn2/scripts/tutorials/deep_trainer/readme.txt:

This is not the best-supported tutorial in pylearn2, in part because most of
the active pylearn2 developers are no longer researching stacked layerwise
pretraining.

Just curious ... what is the active research area for the pylearn2 developers for unsupervised feature learning right now?

Thanks!

-Tom

--
You received this message because you are subscribed to the Google Groups "pylearn-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pylearn-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[David Warde-Farley]

Layerwise pretraining has limited (though still nonzero) utility in light of more recent work, but even much of the unsupervised learning work had moved towards joint training strategies over layerwise strategies. The 4 researchers who contribute the most to pylearn2 currently are generally not working on unsupervised feature learning at the moment (one is working on RBMs but in a particular somewhat exotic context).

There is some autoencoder stuff in the library that needs to be updated to work with new pylearn2 features (or, more likely, removed and replaced with code based on the existing and up to date MLP implementation), and the DBM implementation is quite general, but it's complicated and not extensively documented yet last I looked. We welcome contributions, though for new contributors it's best if they outline their plans on the mailing list so that they don't waste a lot of time on a large contribution that will be rejected because it neglects some key consideration the author didn't think of.

To unsubscribe from this group and stop receiving emails from it, send an email to pylearn-user...@googlegroups.com.

[Tom Szumowski]

David,

Layerwise pretraining has limited (though still nonzero) utility in light of more recent work, but even much of the unsupervised learning work had moved towards joint training strategies over layerwise strategies

I've been using the Representation Learning paper (http://arxiv.org/abs/1206.5538) as a foundation for searching for more relevant research in the area of unsupervised learning. Section 10.2 is labeled "Joint Training of Deep Boltzmann Machines". Is that what you mean by "joint training strategy" above ... for unsupervised applications?

I know it seems this thread is digressing from pylearn2 itself. But in particular, I'm interested in learning more about creating effective, reduced/sparse representations of complex data (unsupervised) using deep learning methods from literature. Where pylearn2 comes into play is I'm currently going through the tutorials/codebase to see what would be the most appropriate tool for this application before I diverge too far. And it sounds like I started diverging towards older research. :)

Thanks again for the advice!

-Tom

[Ian Goodfellow]

Pylearn2 has some code for joint training of DBMs that I wrote for my NIPS 2013 paper "Multi-Prediction Deep Boltzmann Machines." 

[Tom Szumowski]

Ian

Thanks I'll check that out!

You received this message because you are subscribed to a topic in the Google Groups "pylearn-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylearn-users/FysRzk6KIO4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylearn-user...@googlegroups.com.

[Kyle Kastner]

Isn't Dr. Bengio's et. al's GSN work (http://arxiv.org/abs/1306.1091) unsupervised? Looks like gsn.py was added a while back, though I don't know anything else about it.

[Ian Goodfellow]

Yes, but the Pylearn2 code for it was written by an intern who is gone now. None of the regular maintainers of Pylearn2 are working on GSNs. 

[David]

Replying to an old thread about unsupervised learning implementations in pylearn2.

Is it still true that none of the unsupervised learning code is actively maintained? I am interested in doing unsupervised feature learning in a vision application.