Building the TensorFlow MLIR components via CMake?

[Ben Vanik]

It looks like sometime this summer TensorFlow removed its cmake build support. This means that the tensorflow/compiler/mlir/... files are difficult to build under cmake (compounded by the fact that they include some core tensorflow headers that pull in eigen and other deps). Has anyone tried to build the tensorflow/compiler/mlir/ files (like tflite, xla, etc) without using bazel, or with some nasty mashup of bazel+cmake?

[Ben Vanik]

(whatever I'm able to get working I'll publish to 

https://github.com/google/iree/ so others can use it, unless I'm the only one crazy enough to do so ;)

[Jacques Pienaar]

I thought you would either want to just use bazel (and then there is existing rules) or just use cmake (and then you don't have rules TF/TFLite/XLA side). What would bazel + cmake get you that bazel alone won't? (but something like https://github.com/bazelbuild/rules_foreign_cc could allow this). And unfortunately no cmake build TF side. Generating cmake from bazel build may be possible, but I don't know of any project that does that unfortunately.

-- Jacques

--
You received this message because you are subscribed to the Google Groups "MLIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlir+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/mlir/CAOB_hFRtVQ_Lce%3DTi_jrT_xDsdGHc7FTN4rJMzf2_yRJmnCMUA%40mail.gmail.com.

[Ben Vanik]

My thought was to try to make it easier for people to integrate projects that are middlewareish without forcing them to also have bazel (and require a fully functioning tensorflow checkout, which can sometimes be tricky), as cmake is the defacto standard and quite ubiquitous (especially for edge platforms/embedded devices, where bazel crosstool configs can be pretty complex to setup).

I was planning on having a "pure" MLIR build that does not rely on tensorflow and only requires LLVMSupport/MLIR to make builds fast and stable and then wanted to optionally enable tensorflow import with a flag, but it seems to be an eat-the-world kind of thing. But, maybe it's fine to basically shell out to bazel to build tensorflow (cmake equivalent of the foreign cc rules) if the user wants to ingest tf/xla models - the thing that may get tricky there is dependency tracking for build invalidation and compile_commands for clangd/vscode/etc as they generally only accept one input.

But, typing all that out makes me realize I could probably get by with shelling out to bazel for now and move on to more fun MLIR work, so I may give that a go :)

I did notice that tensorflow has a copy of MLIR embedded within it (meaning that revisions don't line up between tensorflow/tensorflow/third_party/mlir vs. tensorflow/mlir) - if that's the long term plan then maybe I should be flipping around my dependency and only depending on tensorflow, instead of trying to get MLIR on its own to work inside of TF. Thoughts?

[Mehdi AMINI]

On Tue, Sep 24, 2019 at 4:31 PM 'Ben Vanik' via MLIR <ml...@tensorflow.org> wrote:

My thought was to try to make it easier for people to integrate projects that are middlewareish without forcing them to also have bazel (and require a fully functioning tensorflow checkout, which can sometimes be tricky), as cmake is the defacto standard and quite ubiquitous (especially for edge platforms/embedded devices, where bazel crosstool configs can be pretty complex to setup).

I was planning on having a "pure" MLIR build that does not rely on tensorflow and only requires LLVMSupport/MLIR to make builds fast and stable and then wanted to optionally enable tensorflow import with a flag, but it seems to be an eat-the-world kind of thing. But, maybe it's fine to basically shell out to bazel to build tensorflow (cmake equivalent of the foreign cc rules) if the user wants to ingest tf/xla models - the thing that may get tricky there is dependency tracking for build invalidation and compile_commands for clangd/vscode/etc as they generally only accept one input.

But, typing all that out makes me realize I could probably get by with shelling out to bazel for now and move on to more fun MLIR work, so I may give that a go :)

I did notice that tensorflow has a copy of MLIR embedded within it (meaning that revisions don't line up between tensorflow/tensorflow/third_party/mlir vs. tensorflow/mlir) - if that's the long term plan then maybe I should be flipping around my dependency and only depending on tensorflow, instead of trying to get MLIR on its own to work inside of TF. Thoughts?

The "long term plan" is that MLIR will be integrated in the LLVM monorepo, the copy in TensorFlow will be removed and the LLVM clone in TensorFlow will be used instead (and it'll provide MLIR at this point).

-- 

Mehdi

 

On Tue, Sep 24, 2019 at 3:54 PM Jacques Pienaar <jpie...@google.com> wrote:

I thought you would either want to just use bazel (and then there is existing rules) or just use cmake (and then you don't have rules TF/TFLite/XLA side). What would bazel + cmake get you that bazel alone won't? (but something like https://github.com/bazelbuild/rules_foreign_cc could allow this). And unfortunately no cmake build TF side. Generating cmake from bazel build may be possible, but I don't know of any project that does that unfortunately.

-- Jacques

On Tue, Sep 24, 2019 at 3:13 PM 'Ben Vanik' via MLIR <ml...@tensorflow.org> wrote:

(whatever I'm able to get working I'll publish to 

https://github.com/google/iree/ so others can use it, unless I'm the only one crazy enough to do so ;)

On Tue, Sep 24, 2019 at 14:35 Ben Vanik <benv...@google.com> wrote:

It looks like sometime this summer TensorFlow removed its cmake build support. This means that the tensorflow/compiler/mlir/... files are difficult to build under cmake (compounded by the fact that they include some core tensorflow headers that pull in eigen and other deps). Has anyone tried to build the tensorflow/compiler/mlir/ files (like tflite, xla, etc) without using bazel, or with some nasty mashup of bazel+cmake?

--
You received this message because you are subscribed to the Google Groups "MLIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlir+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/mlir/CAOB_hFRtVQ_Lce%3DTi_jrT_xDsdGHc7FTN4rJMzf2_yRJmnCMUA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "MLIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlir+uns...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/mlir/CAOB_hFT8rMFH1tVGj2Z6wMU2FMYoP08GAxvoO__OAaeZ6wNLDA%40mail.gmail.com.