Re: tensorflow

[Sameer Deshmukh]

Can you post this on the mailing list with all references? As in you can link to the code where you're saying that it's implemented in Python.

This projects aim is to make a Ruby wrapper over tensor flow. Diving into Python is not of much consequence right now. It can be a later project.

Regards,

Sameer Deshmukh

On 13-May-2016, at 19:21, Ankur Yadav <ankur...@gmail.com> wrote:

Hi Sameer,

I have gone through the blog post but it seems to me that given deliverable is tough to achieve for a single student in span of 3 months.

Most of the usable apis are different algorithms that are present in python only and are not present in C++ apis. So, I think most of the abstraction that people generally use for machine learning is still required to be developed in Ruby, although we can use python apis as the reference but I still think it will require time more than 3 months.

If you guys have any different strategy or any suggestion to tackle this problem in 3 month then please do suggest please.

I would suggest to have a chat on different issues either on irc or gitter that can boost this project.

Regards,

Ankur Yadav

http://ankuryadav.in

On 13 May 2016 at 01:18, Sameer Deshmukh <sameer.d...@gmail.com> wrote:

Hi,

Did you guys go over the blog post?

I'd like to hear ideas from your end.

Regards,

Sameer Deshmukh

[Sameer Deshmukh]

Also, why is SWIG the best way to develop the Ruby wrapper? Why not FFI or native C code? Any specific reasons?

Regards,

Sameer Deshmukh

[John Woods]

Well, I do know TensorFlow recommends SWIG, because this is what they use for their other interfaces.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ankur Yadav]

Hi all,

I am still reviewing the code and trying to relate it with the whitepaper.

I will soon post my new proposal once I get an idea about how I would like to tackle this project.

I think i will post my primary proposal in 2-3 days for review from the community.

Regards,

Ankur Yadav

[Ankur Yadav]

Hi Sameer,

 

In issue https://github.com/tensorflow/tensorflow/issues/388  one of the tensorflow member girving suggested 3 different levels of bindings.

In which he suggest to start off with (1) (Bindings that do not know about specific ops, but can create graphs "manually" or load them from GraphDefs, evaluate graphs etc. There's hopefully nothing blocking that) 

So, for this we have to build and load graph using python and then we can evaluate and work on graphs using C++ api wrappers in Ruby.

He also says "We currently have a lot of logic in pure Python, including both per-op sugar and key features such as automatic differentiation. We will likely eventually move this to C++, but until then it will be difficult to capture this functionality in rust bindings without duplicating a ton of effort. Unfortunately, automatic differentiation in particular is (mostly) necessary if you want to train models."

Also I have noted that most of people who are making bindings for other languages are using binding level (1) as suggested by girving.

Regards,

Ankur Yadav

[SoonHin Khor]

Ankur,

Thanks for sharing this!

khor

[Sameer Deshmukh]

What exactly do you mean by 'graphs'? Are they matplotlib graphs or something more tensor flow specific? It's a term being thrown around a lot but I can't really get a hang of it.

Why not port the graph functions to Ruby? Ruby-Python bridges are rather slow and clumsy.

Regards,

Sameer Deshmukh

On 18-May-2016, at 10:01, Ankur Yadav <ankur...@gmail.com> wrote:

Hi Sameer,

Yes, It seems that it would be possible to implement in 3 month period if we only concentrate on MNIST tutorial.

We can develop a generic C++ api binding in Ruby and use it wherever required in MNIST.

For the other part where python apis are required we can make ruby functions for that purpose.

For some python code which cannot be converted to ruby at this time we can use rubypython bindings to do that job, for example for making graphs and writing graphs.

So, if we proceed with this plan then I think my previous proposal would be applicable here.

Regards,

Ankur Yadav

On 18 May 2016 at 09:15, Sameer Deshmukh <sameer.d...@gmail.com> wrote:

Is a 3 month period enough to implement the MNIST tutorial? Which levels does that use?

Regards,

Sameer Deshmukh

[SoonHin Khor]

Sorry. I have used the term graph a bit as well in the other emails.

Graph is a term used to refer to the Tensorflow model; a Tensorflow model consists mainly of nodes and edges.
Nodes can be constants, variables, placeholders, etc. in vector form, OR operations, e.g., matrix multiplication, softmax, etc.

Edges just connect nodes together.

E.g. to model, y = W.x, We use two nodes W, and x that has edges to matrix multiplcation node. Another edge leaving the matrix multiplication node would be the output, which can be chained to other nodes.

[Sameer Deshmukh]

This looks like critical tensor flow functionality. Are you sure it's available only in Python? Even if it is, the Ruby wrapper should have this because of its critical nature. Tensor flow basically works by representing your computation as nodes and tensors right? I'd say go ahead with MNIST and port Python code to Ruby. 

Of course I'm open to comments from the others.

Regards,

Sameer Deshmukh

[John Woods]

NMatrix is well-suited to the representation of graphs. You'd use :dense for graphs where every node connects to nearly every other node. You'd use :list and :yale for sparser graphs (with the latter being preferable when you want to do multiplicative computations on those graphs).

I'd prefer we either abstract Python code to C or port it. Ruby–Python bridges = bad news bears.

[Pjotr Prins]

The only reasonable way to pass data between Ruby and Python programs
is through a pipe - which goes one way only. Sometimes that is good
enough, say for transforming data.

Any other type of bridge, I agree, is a rotten idea. Having the two
interpreters in one place, even over a network interface, is bound to
cause unreasonably headaches.

Pj.

On Wed, May 18, 2016 at 03:45:11PM +0000, John Woods wrote:
> NMatrix is well-suited to the representation of graphs. You'd use :dense
> for graphs where every node connects to nearly every other node. You'd use
> :list and :yale for sparser graphs (with the latter being preferable when
> you want to do multiplicative computations on those graphs).
> I'd prefer we either abstract Python code to C or port it. Ruby–Python
> bridges = bad news bears.
> On Wed, May 18, 2016 at 7:38 AM Sameer Deshmukh
> <[1]sameer.d...@gmail.com> wrote:
>
> This looks like critical tensor flow functionality. Are you sure it's
> available only in Python? Even if it is, the Ruby wrapper should have
> this because of its critical nature. Tensor flow basically works by
> representing your computation as nodes and tensors right? I'd say go
> ahead with MNIST and port Python code to Ruby. 
> Of course I'm open to comments from the others.
>
> Regards,
> Sameer Deshmukh

> On 18-May-2016, at 12:12, SoonHin Khor <[2]khor.s...@gmail.com>

> wrote:
>
> Sorry. I have used the term graph a bit as well in the other emails.
> Graph is a term used to refer to the Tensorflow model; a Tensorflow
> model consists mainly of nodes and edges.
> Nodes can be constants, variables, placeholders, etc. in vector form,
> OR operations, e.g., matrix multiplication, softmax, etc.
> Edges just connect nodes together.
> E.g. to model, y = W.x, We use two nodes W, and x that has edges to
> matrix multiplcation node. Another edge leaving the matrix
> multiplication node would be the output, which can be chained to other
> nodes.
> On Wed, May 18, 2016 at 2:39 PM, Sameer Deshmukh
> <[3]sameer.d...@gmail.com> wrote:
>
> What exactly do you mean by 'graphs'? Are they matplotlib graphs or
> something more tensor flow specific? It's a term being thrown around
> a lot but I can't really get a hang of it.
> Why not port the graph functions to Ruby? Ruby-Python bridges are
> rather slow and clumsy.
>
> Regards,
> Sameer Deshmukh

> On 18-May-2016, at 10:01, Ankur Yadav <[4]ankur...@gmail.com>

> wrote:
>
> Hi Sameer,
> Yes, It seems that it would be possible to implement in 3 month
> period if we only concentrate on MNIST tutorial.
> We can develop a generic C++ api binding in Ruby and use it
> wherever required in MNIST.
> For the other part where python apis are required we can make ruby
> functions for that purpose.
> For some python code which cannot be converted to ruby at this
> time we can use rubypython bindings to do that job, for example
> for making graphs and writing graphs.
> So, if we proceed with this plan then I think my previous proposal
> would be applicable here.
> Regards,
> Ankur Yadav
> On 18 May 2016 at 09:15, Sameer Deshmukh

> <[5]sameer.d...@gmail.com> wrote:
>
> Is a 3 month period enough to implement the MNIST tutorial?
> Which levels does that use?
>
> Regards,
> Sameer Deshmukh

> On 18-May-2016, at 03:15, Ankur Yadav <[6]ankur...@gmail.com>
> wrote:
>
> Hi Sameer,
>  
> In
> issue [7]https://github.com/tensorflow/tensorflow/issues/388  one
> of the tensorflow member [8]girving suggested 3 different

> levels of bindings.
> In which he suggest to start off with (1) (Bindings that do
> not know about specific ops, but can create graphs "manually"
> or load them from GraphDefs, evaluate graphs etc. There's
> hopefully nothing blocking that) 
> So, for this we have to build and load graph using python and
> then we can evaluate and work on graphs using C++ api wrappers
> in Ruby.
> He also says "We currently have a lot of logic in pure Python,
> including both per-op sugar and key features such as automatic
> differentiation. We will likely eventually move this to C++,
> but until then it will be difficult to capture this
> functionality in rust bindings without duplicating a ton of
> effort. Unfortunately, automatic differentiation in particular
> is (mostly) necessary if you want to train models."
> Also I have noted that most of people who are making bindings
> for other languages are using binding level (1) as suggested

> by [9]girving.

> Regards,
> Ankur Yadav
>
> --
> You received this message because you are subscribed to the Google
> Groups "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [10]sciruby-dev...@googlegroups.com.
> For more options, visit [11]https://groups.google.com/d/optout.

>
> --
> You received this message because you are subscribed to the Google Groups
> "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to [12]sciruby-dev...@googlegroups.com.
> For more options, visit [13]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:sameer.d...@gmail.com
> 2. mailto:khor.s...@gmail.com
> 3. mailto:sameer.d...@gmail.com
> 4. mailto:ankur...@gmail.com
> 5. mailto:sameer.d...@gmail.com
> 6. mailto:ankur...@gmail.com
> 7. https://github.com/tensorflow/tensorflow/issues/388
> 8. https://github.com/girving
> 9. https://github.com/girving
> 10. mailto:sciruby-dev...@googlegroups.com
> 11. https://groups.google.com/d/optout
> 12. mailto:sciruby-dev...@googlegroups.com
> 13. https://groups.google.com/d/optout

--

[Ankur Yadav]

I think the main distinguishing feature of tensorflow is the way it stores its graph and how to changes its graph according to the platform on which we want to run tensorflow. 

So, like for single processor or multi processor or even for distributed computation over a network it simply abstract the way out from user so that programmers can only think about building models and they tensorflow will make sure that all tensors gets executed cross platform.

I think using NMatrix for graph will completely change the platform.

I was suggesting ruby-python bridge just as an temporary solution and as an option.

[Ankur Yadav]

I think let me focus on public C-APIs first and once it is done I will try to focus on how to create graphs as currently I still do not have very deep understanding of the framework and I think I will be able to do it after I gain some experience in porting the APIs.

Regards,

Ankur Yadav