How could I understand TensorFlow source code?

5,123 views
Skip to first unread message

Bowen Yu

unread,
Sep 1, 2016, 10:36:01 AM9/1/16
to Discuss
Hi, all! 

I'm a PhD student. I'm interested in optimizing the performance of TensorFlow, as it's close to my research interest. I have read the TensorFlow white paper and have had a general understanding of TensorFlow design. But I want more specific details about its implementation by reading source code. However, My attempt to reading its enormous amount of source code seems very inefficient. TensorFlow is too complex for me, and I have no idea where to start.

How do you understand TensorFlow's source code?

Pr Si

unread,
Sep 1, 2016, 11:06:34 AM9/1/16
to Bowen Yu, Discuss
I think you should take some program (for any task) where tensorflow is used.
It should not be that difficult to understand the Program's source code.
Thus, you can have some understanding of tensorflow concepts.

This can be a starting point for your understanding.
Similarly, you can take more and more programs (Whatever Problem areas are being addressed till date) 
and understand the associated tensorflow concepts.

In this way, you should be able to understand quite a lot of tensorflow concepts.

I hope it helps !!

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/4967fb7e-5267-410c-b26d-c7440c760d7c%40tensorflow.org.



--
If the next generation is to face the future with zest and self-confidence,
we must educate them to be original as well as competent.

Bowen Yu

unread,
Sep 1, 2016, 11:49:14 AM9/1/16
to Discuss
Thank you for your reply! 

I mean understand the implementation of TensorFlow in detail because I want to modify and contribute the code and do experiment.
For example, if I want to know what node-placement algorithms does TensorFlow used, or how does TensorFlow execute the computing, I have to dig into TensorFlow's source code. 

Applications often use higher-level front-end, like Python. However, it is the framework that actually do the work.

Maybe documentation or information about the responsibility of each module and the relationship among modules could help.

在 2016年9月1日星期四 UTC+8下午11:06:34，Pr Si写道：
I think you should take some program (for any task) where tensorflow is used.
It should not be that difficult to understand the Program's source code.
Thus, you can have some understanding of tensorflow concepts.

This can be a starting point for your understanding.
Similarly, you can take more and more programs (Whatever Problem areas are being addressed till date) 
and understand the associated tensorflow concepts.

In this way, you should be able to understand quite a lot of tensorflow concepts.

I hope it helps !!
On Thu, Sep 1, 2016 at 10:36 AM, Bowen Yu <ybw...@gmail.com> wrote:
Hi, all! 

I'm a PhD student. I'm interested in optimizing the performance of TensorFlow, as it's close to my research interest. I have read the TensorFlow white paper and have had a general understanding of TensorFlow design. But I want more specific details about its implementation by reading source code. However, My attempt to reading its enormous amount of source code seems very inefficient. TensorFlow is too complex for me, and I have no idea where to start.

How do you understand TensorFlow's source code?

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

Eugene Brevdo

unread,
Sep 1, 2016, 11:59:59 AM9/1/16
to Bowen Yu, Discuss
You're probably mostly interested in the executor and direct session code:


as a place to start.  Of course, tons of things affect runtime performance, including kernel implementations of particular ops.  This is a fairly low level part of the code base.  Other places where optimization has been important in the past is in the thread pool code.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/67bedf8d-c0c4-4232-9197-d98eac6f4d21%40tensorflow.org.

Sam A

unread,
Sep 1, 2016, 12:10:11 PM9/1/16
to Eugene Brevdo, Bowen Yu, Discuss
This discussion on the structure of the TensorFlow code base should be helpful:


Probably the highlight from that conversation is the description of each subdirectory from Mr. Eugene Brevdo:

core/ contains the main C++ code and runtimes.

core/ops/ contains the "signatures" of the operations
core/kernels/ contains the "implementations" of the operations (including CPU and CUDA kernels)
core/framework/ contains the main abstract graph computation and other useful libraries
core/platform/ contains code that abstracts away the platform and other imported libraries (protobuf, etc)

TensorFlow relies heavily on the Eigen library for both CPU and GPU calculations.  Though some GPU kernels are implemented directly with CUDA code.

bazel builds certain C++ code using gcc/clang, and certain CUDA code (files with extension .cu.cc) with nvcc.

python/ops/ contain the core python interface
python/kernel_tests/ contain the unit tests and lots of example code
python/framework/ contains the python abstractions of graph, etc, a lot of which get serialized down to proto and/or get passed to swigged session calls.
python/platform/ is similar to the C++ platform, adding lightweight wrappers for python I/O, unit testing, etc.

contrib/*/ directories generally mimic the root tensorflow path (i.e., they have core/ops/, etc)

As well as this portion of the core build script:


To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/CAJRAvvbyW2GZqcc%3DBfu1rY0U9T%3DA5_WK%3DkwYWsBKLF6GCv4TmQ%40mail.gmail.com.

Zhenqing Hu

unread,
Mar 6, 2018, 9:27:25 AM3/6/18
to Discuss
It's a clear description for the structure of the source code, thanks!

在 2016年9月1日星期四 UTC-4下午12:10:11，Sam Abrahams写道：

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
Reply all
Reply to author
Forward
0 new messages