beginning deep learning with 500 lines of Julia

[Deniz Yuret]

KUnet.jl (beginning deep learning with 500 lines of Julia) is out with its alpha release.  Only the basic functionality is in place (i.e. backprop with relu, softmax, sgd, momentum, nesterov, adagrad, dropout, l1-l2 regularization etc.) but the GPU functionality is in, its speed is competitive with Caffe, and I think convolutional and recurrent nets can be added without too much effort.  I wrote a blog post about the code structure and there is some basic documentation.  You can send me suggestions for improvement (both in coding style and new functionality) using comments to the blog post, or using issues or pull requests on GitHub.

I tried to make the code (cpu/gpu) generic and high level.  Getting the same code working on the GPU and the CPU in Julia proved to be a bit challenging and showed that both a more standard treatment of CPU and GPU arrays, and a standard syntax for in-place operations would be welcome additions to the language.  I'd like to thank Tim Holy (CUDArt), Nick Henderson (CUBLAS), and Simon Byrne (InplaceOps) for their generous help.

best,

deniz

[Yuuki Soho]

Nice article, it reminded me of my machine learning courses :)

The only comment I have is maybe use a bit more explicit names, like forw -> forward, xtst -> xtest

[Zhixuan Yang]

Nice article and code indeed! I will try this in my project to see how well it works. :-) 

在 2015年2月28日星期六 UTC+8下午11:19:18，Deniz Yuret写道：

[Jon Norberg]

Remarkably good idea, thanks for sharing. I like the clean readable code but efficient if needed idea. It would be amazing if the recent developments could be added as you hint at for people to learn.

many thanks

[Simon Danisch]

I'm a little out of  sync with machine learning but this looks really awesome =)

I do hope as well, that we move forward with a unified array interface in JuliaGPU!