[llvm-dev] Google’s TensorFlow team would like to contribute MLIR to the LLVM Foundation
Chris Lattner via llvm-dev
The TensorFlow team at Google has been leading the charge to build a new set of compiler infrastructure, known as the MLIR project. The initial focus has been on machine learning infrastructure, high performance accelerators, heterogeneous compute, and HPC-style computations. That said, the implementation and design of this infrastructure is state of the art, is not specific to these applications, and is already being adopted (e.g.) by the Flang compiler. If you are interested in learning more about MLIR and the technical design, I’d encourage you to look at the MLIR Keynote and Tutorial at the last LLVM Developer Meeting.
MLIR is already open source on GitHub, and includes a significant amount of code in two repositories. “MLIR Core” is located in github/tensorflow/mlir, including an application independent IR, the code generation infrastructure, common graph transformation infrastructure, declarative operation definition and rewrite infrastructure, polyhedral transformations etc. The primary TensorFlow repository at github/tensorflow/tensorflow contains TensorFlow-specific functionality built using MLIR Core infrastructure.
In discussions with a large number of industry partners, we’ve achieved consensus that it would be best to build a shared ML compiler infrastructure under a common umbrella with well known neutral governance. As such, we’d like to propose that MLIR Core join the non-profit LLVM Foundation as a new subproject! We plan to follow the LLVM Developer Policy, and have been following an LLVM-style development process from the beginning - including all relevant coding and testing styles, and we build on core LLVM infrastructure pervasively.
We think that MLIR is a nice complement to existing LLVM functionality, providing common infrastructure for higher level optimization and transformation problems, and dovetails naturally with LLVM IR optimizations and code generation. Please let us know if you have any thoughts, questions, or concerns!
-Chris
Finkel, Hal J. via llvm-dev
Hi, Chris, et al.,
I support adding MLIR as an LLVM subproject. Here are my thoughts:
1. MLIR uses LLVM. LLVM is one of the MLIR dialects, MLIR is compiler infrastructure, and it fits as a natural part of our ecosystem.
2. As a community, we have a lot of different LLVM frontends, many of which have their own IRs on which higher-level transformations are performed. We don't currently offer much, in terms of infrastructure, to support the development of these pre-LLVM transformations.
MLIR provides a base on which many of these kinds of implementations can be constructed, and I believe that will add value to the overall ecosystem.
3. As a specific example of the above, the current development of the new Flang compiler depends on MLIR. Flang is becoming a subproject of LLVM and MLIR should be part of LLVM.
4. The MLIR project has developed capabilities, such as for the analysis of multidimensional loops, that can be moved into LLVM and used by both LLVM- and MLIR-level transformations. As we work to improve LLVM's capabilities in loop optimizations, leveraging continuing work to improve MLIR's loop capabilities in LLVM as well will benefit many of us.
5. As a community, we have been moving toward increasing support for heterogeneous computing and accelerators (and given industry trends, I expect this to continue), and MLIR can facilitate that support in many cases (although I expect we'll see further enhancements in the core LLVM libraries as well).
That all having been said, I think that it's going to be very important to develop some documentation on how a frontend author looking to use LLVM backend technology, and a developer looking to implement different kinds of functionality, might reasonably choose whether to target or enhance MLIR components, LLVM components, or both. I expect that this kind of advice will evolve over time, but I'm sure we'll need it sooner rather than later.
Thanks again,
Hal
_______________________________________________ LLVM Developers mailing list llvm...@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- Hal Finkel Lead, Compiler Technology and Programming Languages Leadership Computing Facility Argonne National Laboratory
Renato Golin via llvm-dev
Maintenance wise, I'm expecting the existing community to move into
LLVM (if not all in already), so I don't foresee any additional costs.
Though, Hal's points are spot on...
On Mon, 9 Sep 2019 at 18:47, Finkel, Hal J. via llvm-dev
<llvm...@lists.llvm.org> wrote:
> 3. As a specific example of the above, the current development of the new Flang compiler depends on MLIR.
Who knows, one day, Clang can, too! :)
> 5. As a community, we have been moving toward increasing support for heterogeneous computing and accelerators (and given industry trends, I expect this to continue), and MLIR can facilitate that support in many cases (although I expect we'll see further enhancements in the core LLVM libraries as well).
Yes, and yes! MLIR can become a simpler entry point into LLVM, from
other languages, frameworks and optimisation plugins. A more abstract
representation and a more stable IR generation from it, could make
maintenance of external projects much easier than direct connections
of today. This could benefit research as much as enterprise, and by
consequence, the LLVM project.
> That all having been said, I think that it's going to be very important to develop some documentation on how a frontend author looking to use LLVM backend technology, and a developer looking to implement different kinds of functionality, might reasonably choose whether to target or enhance MLIR components, LLVM components, or both. I expect that this kind of advice will evolve over time, but I'm sure we'll need it sooner rather than later.
Right, I'm also worried that it's too broad in respect to what it can
do on paper, versus what LLVM can handle on code.
With MLIR as a separate project, that point is interesting, at most.
When it becomes part of the LLVM umbrella, then we need to make sure
that MLIR and LLVM IR interact within known boundaries and expected
behaviour.
I'm not saying MLIR can't be used for anything else after the move,
just saying that, by being inside the repo, and maintained by our
community, LLVM IR would end up as the *primary* target, and there
will be a minimum stability/functionality requirements.
But perhaps more importantly, as Hal states clearly, is the need for
an official specification, similar to the one for LLVM IR, as well as
a formal document with the expected semantics into LLVM IR. Sooner,
indeed.
cheers,
--renato
Chris Lattner via llvm-dev
Sjoerd Meijer via llvm-dev
Sent: 09 September 2019 22:22
To: Renato Golin <reng...@gmail.com>
Cc: llvm-dev <llvm...@lists.llvm.org>; Reid Tatge <ta...@google.com>; Mehdi Amini <ami...@google.com>; Tatiana Shpeisman <shpe...@google.com>
Subject: Re: [llvm-dev] Google’s TensorFlow team would like to contribute MLIR to the LLVM Foundation
Renato Golin via llvm-dev
> Including a bunch of content, eg a full langref doc:
> https://github.com/tensorflow/mlir/blob/master/g3doc/LangRef.md
Thanks Chris, that looks awesome!
This one could perhaps be improved with time:
https://github.com/tensorflow/mlir/blob/master/g3doc/ConversionToLLVMDialect.md
Which I think was Hal's point. If we had a front-end already using it
in tree, we could be a bit more relaxed with the conversion
specification.
I remember when I did the EDG bridge to LLVM, I mostly repeated
whatever Clang was doing, "bug-for-bug". :)
A cheeky request, perhaps, for the Flang people: they could help with
that document on what they have learned using MLIR as a front-end into
LLVM IR.
We get some common patterns written down, but also we get to review
their assumptions earlier, and make sure that both Flang and MLIR
co-evolve into something simpler.
Mehdi Amini via llvm-dev
Overall, I think it will be a good move.
Maintenance wise, I'm expecting the existing community to move into
LLVM (if not all in already), so I don't foresee any additional costs.
Though, Hal's points are spot on...
On Mon, 9 Sep 2019 at 18:47, Finkel, Hal J. via llvm-dev
<llvm...@lists.llvm.org> wrote:
> 3. As a specific example of the above, the current development of the new Flang compiler depends on MLIR.
Who knows, one day, Clang can, too! :)
> 5. As a community, we have been moving toward increasing support for heterogeneous computing and accelerators (and given industry trends, I expect this to continue), and MLIR can facilitate that support in many cases (although I expect we'll see further enhancements in the core LLVM libraries as well).
Yes, and yes! MLIR can become a simpler entry point into LLVM, from
other languages, frameworks and optimisation plugins. A more abstract
representation and a more stable IR generation from it, could make
maintenance of external projects much easier than direct connections
of today. This could benefit research as much as enterprise, and by
consequence, the LLVM project.
> That all having been said, I think that it's going to be very important to develop some documentation on how a frontend author looking to use LLVM backend technology, and a developer looking to implement different kinds of functionality, might reasonably choose whether to target or enhance MLIR components, LLVM components, or both. I expect that this kind of advice will evolve over time, but I'm sure we'll need it sooner rather than later.
Right, I'm also worried that it's too broad in respect to what it can
do on paper, versus what LLVM can handle on code.
With MLIR as a separate project, that point is interesting, at most.
When it becomes part of the LLVM umbrella, then we need to make sure
that MLIR and LLVM IR interact within known boundaries and expected
behaviour.
I'm not saying MLIR can't be used for anything else after the move,
just saying that, by being inside the repo, and maintained by our
community, LLVM IR would end up as the *primary* target, and there
will be a minimum stability/functionality requirements.
The lines I highlighted are defining a GPU kernel, wrapped in a "gpu.launch" operation. The `mlir-cuda-runner` is a command line tool that tests will run passes to separate the kernel GPU code from the host code, and emit the LLVM IR in two separate LLVM modules: one for the GPU kernel (using the NVPTX backend) and another one for the host. Then everything is ran through a JIT (assuming you have CUDA and a compatible GPU installed).
Chris Lattner via llvm-dev
> On Sep 9, 2019, at 3:39 PM, Renato Golin via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> On Mon, 9 Sep 2019 at 22:22, Chris Lattner <clat...@google.com> wrote:
>> Including a bunch of content, eg a full langref doc:
>> https://github.com/tensorflow/mlir/blob/master/g3doc/LangRef.md
>
> Thanks Chris, that looks awesome!
>
> This one could perhaps be improved with time:
> https://github.com/tensorflow/mlir/blob/master/g3doc/ConversionToLLVMDialect.md
>
> Which I think was Hal's point. If we had a front-end already using it
> in tree, we could be a bit more relaxed with the conversion
> specification.
Don’t worry, Flang is coming soon :-).
In all seriousness, if you didn’t notice, the Flang team is planning to give a talk at LLVMDev in a month or so about Flang + MLIR. I’d also love to see a round table or other discussion about MLIR integration at the event.
The topic of Clang generating MLIR is more sensitive and I think it is best broached as a separate conversation, one motivated with data. I think that Clang generating MLIR can be a hugely positive thing (witness the explosion of recent proposals for LLVM IR extensions that are easily handled with MLIR) but it seems more conservative and logical to upgrade the existing Clang “CFG" representation to use MLIR first. This brings simple and measurable improvements to the reliability, accuracy, and generality of the data flow analyses and the Clang Static Analyzer, without introducing a new step that could cause compile-time regressions. Iff that goes well, we could consider the use of MLIR in the main compilation flow.
In any case, I hope that "Clang adoption" is not considered to be a blocker for MLIR to be adopted as part of the LLVM project. This hasn’t been a formal or historical requirement for new LLVM subprojects, and I’d like to make sure we don’t put undue adoption pressure on Clang - it is important that we are deliberate about each step and do the right (data driven) thing for the (huge) Clang community.
-Chris
Renato Golin via llvm-dev
> In all seriousness, if you didn’t notice, the Flang team is planning to give a talk at LLVMDev in a month or so about Flang + MLIR. I’d also love to see a round table or other discussion about MLIR integration at the event.
Ah, the title was just "Flang update", I didn't check the abstract.
Looking forward to it.
> The topic of Clang generating MLIR is more sensitive and I think it is best broached as a separate conversation, one motivated with data. I think that Clang generating MLIR can be a hugely positive thing (witness the explosion of recent proposals for LLVM IR extensions that are easily handled with MLIR) but it seems more conservative and logical to upgrade the existing Clang “CFG" representation to use MLIR first. This brings simple and measurable improvements to the reliability, accuracy, and generality of the data flow analyses and the Clang Static Analyzer, without introducing a new step that could cause compile-time regressions. Iff that goes well, we could consider the use of MLIR in the main compilation flow.
Totally agreed!
> In any case, I hope that "Clang adoption" is not considered to be a blocker for MLIR to be adopted as part of the LLVM project. This hasn’t been a formal or historical requirement for new LLVM subprojects, and I’d like to make sure we don’t put undue adoption pressure on Clang - it is important that we are deliberate about each step and do the right (data driven) thing for the (huge) Clang community.
Absolutely.
It doesn't make sense to put artificial orthogonal constraints, when
we know the implementation would raise more questions than answer and
could take years to get right. I'm hoping by adding MLIR first, we'd
have a pretty solid use case and the eventual move by Clang, if any,
would be smoother and more robust.
I agree with this proposal being the first step. I'm also personally
happy with the current level of docs and progress of Flang.
LGTM, thanks! :D
David Greene via llvm-dev
> But perhaps more importantly, as Hal states clearly, is the need for
> an official specification, similar to the one for LLVM IR, as well as
> a formal document with the expected semantics into LLVM IR. Sooner,
> indeed.
+1. There are all kinds of scattered documents on the TensorFlow site
talking about MLIR, the affine dialect, etc. but nothing of the quality
and approachability of LLVM's language reference. I find it difficult
to pull all the pieces together.
Of course by its nature, MLIR doesn't lend itself to concrete semantic
descriptions, though I would expect the affine dialect (and others) to
have documentation on par with the LLVM IR. For MLIR itself, I would
want documentation somewhat less dense than the current BNF-style
specification.
Does the current proposal only cover adding the base MLIR to the LLVM
project, or also the affine dialect and possibly others? The affine
dialect could certainly be quite useful for many projects.
-David
Mehdi AMINI via llvm-dev
On Tue, 10 Sep 2019 at 06:49, Chris Lattner <clat...@google.com> wrote:
> In all seriousness, if you didn’t notice, the Flang team is planning to give a talk at LLVMDev in a month or so about Flang + MLIR. I’d also love to see a round table or other discussion about MLIR integration at the event.
Ah, the title was just "Flang update", I didn't check the abstract.
Mehdi AMINI via llvm-dev
Renato Golin via llvm-dev <llvm...@lists.llvm.org> writes:
> But perhaps more importantly, as Hal states clearly, is the need for
> an official specification, similar to the one for LLVM IR, as well as
> a formal document with the expected semantics into LLVM IR. Sooner,
> indeed.
+1. There are all kinds of scattered documents on the TensorFlow site
talking about MLIR, the affine dialect, etc. but nothing of the quality
and approachability of LLVM's language reference. I find it difficult
to pull all the pieces together.
Of course by its nature, MLIR doesn't lend itself to concrete semantic
descriptions, though I would expect the affine dialect (and others) to
have documentation on par with the LLVM IR.
For MLIR itself, I would
want documentation somewhat less dense than the current BNF-style
specification.
Does the current proposal only cover adding the base MLIR to the LLVM
project, or also the affine dialect and possibly others? The affine
dialect could certainly be quite useful for many projects.
Adve, Vikram Sadanand via llvm-dev
FWIW, I think this could be a very positive step for LLVM and the community. Most of the discussion has been about the details (documentation, in-tree vs. out-of-tree, etc.), but I think there are key bigger picture reasons:
- Obviously, ML languages and frameworks are becoming widespread and a number of teams are investing resources into compilers for them. Having a successful LLVM project that provides good infrastructure for these compilers would be valuable. While the TensorFlow compiler may (or may not) remain a Google project, having a core infrastructure available to the wider community should be super valuable.
- Related point to #1: ML models and even core data types and approaches are evolving rapidly, so there is a lot of research happening on the underlying system infrastructure, from hardware to compilers to languages. If MLIR can become the infrastructure of choice for these research projects (like LLVM did for the scalar and vector compiler world 15 years ago), that would be a big win.
- As Hal said, the LLVM infrastructure has not provided explicit support for high-level analyses and transformations, e.g., loop restructuring, multidimensional arrays, etc. Having a good infrastructure to support these will make a number of languages and hardware targets easier to implement / target.
--Vikram Adve
+ Donald B. Gillies Professor of Computer Science, University of Illinois at Urbana-Champaign
+ Scheduling: Kimberly Baker – kab...@illinois.edu
+ Skype: vikramsadve || Zoom: https://illinois.zoom.us/j/2173900467
+ Home page: http://vikram.cs.illinois.edu
+ Center for Digital Agriculture: https://digitalag.illinois.edu
From:
llvm-dev <llvm-dev...@lists.llvm.org> on behalf of via llvm-dev <llvm...@lists.llvm.org>
Reply-To: "llvm...@lists.llvm.org" <llvm...@lists.llvm.org>, "llvm-dev...@lists.llvm.org" <llvm-dev...@lists.llvm.org>
Date: Monday, September 9, 2019 at 1:58 PM
To: "llvm...@lists.llvm.org" <llvm...@lists.llvm.org>
Subject: llvm-dev Digest, Vol 183, Issue 22
Date: Mon, 9 Sep 2019 17:46:34 +0000
From: "Finkel, Hal J. via llvm-dev" <llvm...@lists.llvm.org>
To: Chris Lattner <clat...@google.com>, llvm-dev
Cc: Reid Tatge <ta...@google.com>, Mehdi Amini <ami...@google.com>,
Tatiana Shpeisman <shpe...@google.com>
Subject: Re: [llvm-dev] Google’s TensorFlow team would like to
contribute MLIR to the LLVM Foundation
Message-ID: <7611fea3-ba64-f587...@anl.gov>
Content-Type: text/plain; charset="utf-8"
Hi, Chris, et al.,
I support adding MLIR as an LLVM subproject. Here are my thoughts:
1. MLIR uses LLVM. LLVM is one of the MLIR dialects, MLIR is compiler infrastructure, and it fits as a natural part of our ecosystem.
2. As a community, we have a lot of different LLVM frontends, many of which have their own IRs on which higher-level transformations are performed. We don't currently offer much, in terms of infrastructure, to support the development of these pre-LLVM transformations. MLIR provides a base on which many of these kinds of implementations can be constructed, and I believe that will add value to the overall ecosystem.
3. As a specific example of the above, the current development of the new Flang compiler depends on MLIR. Flang is becoming a subproject of LLVM and MLIR should be part of LLVM.
4. The MLIR project has developed capabilities, such as for the analysis of multidimensional loops, that can be moved into LLVM and used by both LLVM- and MLIR-level transformations. As we work to improve LLVM's capabilities in loop optimizations, leveraging continuing work to improve MLIR's loop capabilities in LLVM as well will benefit many of us.
5. As a community, we have been moving toward increasing support for heterogeneous computing and accelerators (and given industry trends, I expect this to continue), and MLIR can facilitate that support in many cases (although I expect we'll see further enhancements in the core LLVM libraries as well).
That all having been said, I think that it's going to be very important to develop some documentation on how a frontend author looking to use LLVM backend technology, and a developer looking to implement different kinds of functionality, might reasonably choose whether to target or enhance MLIR components, LLVM components, or both. I expect that this kind of advice will evolve over time, but I'm sure we'll need it sooner rather than later.
Thanks again,
Hal
On 9/9/19 10:30 AM, Chris Lattner via llvm-dev wrote:
Hi all,
The TensorFlow team at Google has been leading the charge to build a new set of compiler infrastructure, known as the MLIR project<https://github.com/tensorflow/mlir>. The initial focus has been on machine learning infrastructure, high performance accelerators, heterogeneous compute, and HPC-style computations. That said, the implementation and design of this infrastructure is state of the art, is not specific to these applications, and is already being adopted (e.g.) by the Flang compiler<https://llvm.org/devmtg/2019-10/talk-abstracts.html#tech19>. If you are interested in learning more about MLIR and the technical design, I’d encourage you to look at the MLIR Keynote and Tutorial at the last LLVM Developer Meeting<http://llvm.org/devmtg/2019-04/>.
MLIR is already open source on GitHub<https://medium.com/tensorflow/mlir-a-new-intermediate-representation-and-compiler-framework-beba999ed18d>, and includes a significant amount of code in two repositories. “MLIR Core” is located in github/tensorflow/mlir<https://github.com/tensorflow/mlir>, including an application independent IR, the code generation infrastructure, common graph transformation infrastructure, declarative operation definition and rewrite infrastructure, polyhedral transformations etc. The primary TensorFlow repository at github/tensorflow/tensorflow<https://github.com/tensorflow/tensorflow/> contains TensorFlow-specific functionality built using MLIR Core infrastructure.
In discussions with a large number of industry partners<https://blog.google/technology/ai/mlir-accelerating-ai-open-source-infrastructure/>, we’ve achieved consensus that it would be best to build a shared ML compiler infrastructure under a common umbrella with well known neutral governance. As such, we’d like to propose that MLIR Core join the non-profit LLVM Foundation as a new subproject! We plan to follow the LLVM Developer Policy<http://llvm.org/docs/DeveloperPolicy.html>, and have been following an LLVM-style development process from the beginning - including all relevant coding and testing styles, and we build on core LLVM infrastructure pervasively.
We think that MLIR is a nice complement to existing LLVM functionality, providing common infrastructure for higher level optimization and transformation problems, and dovetails naturally with LLVM IR optimizations and code generation. Please let us know if you have any thoughts, questions, or concerns!
-Chris
_______________________________________________
LLVM Developers mailing list
--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
David Greene via llvm-dev
> Of course by its nature, MLIR doesn't lend itself to concrete semantic
>> descriptions, though I would expect the affine dialect (and others) to
>> have documentation on par with the LLVM IR.
>
>
> Just last week I had to scout through the affine dialect "LangRef
> <https://github.com/tensorflow/mlir/blob/master/g3doc/Dialects/Affine.md>"
> for something, and I also felt that it is due for a refresh! It seemed a
> bit more than just BNF though, do you have example of what you would like
> to see expanded there?
I was referring to the base MLIR documentation with the BNF comment:
https://github.com/tensorflow/mlir/blob/master/g3doc/LangRef.md
Obviously there's more to it than that but I found this document pretty
dense.
> The current proposal includes all the content of
> https://github.com/tensorflow/mlir/ as-is.
> It does *not* include the TensorFlow specific dialects and other pieces
> here:
> https://github.com/tensorflow/tensorflow/tree/master/tensorflow/compiler/mlir/
Looks great, thanks for making it more clear!
Mehdi Amini via llvm-dev
Mehdi AMINI <joke...@gmail.com> writes:
> Of course by its nature, MLIR doesn't lend itself to concrete semantic
>> descriptions, though I would expect the affine dialect (and others) to
>> have documentation on par with the LLVM IR.
>
>
> Just last week I had to scout through the affine dialect "LangRef
> <https://github.com/tensorflow/mlir/blob/master/g3doc/Dialects/Affine.md>"
> for something, and I also felt that it is due for a refresh! It seemed a
> bit more than just BNF though, do you have example of what you would like
> to see expanded there?
I was referring to the base MLIR documentation with the BNF comment:
https://github.com/tensorflow/mlir/blob/master/g3doc/LangRef.md
Obviously there's more to it than that but I found this document pretty
dense.
Renato Golin via llvm-dev
> The MLIR LangRef corresponds to the former part only, because this is what is common to all dialects. On the other hand, each dialect will need to provide its own LangRef equivalent (for example I linked to the Affine dialect doc before).
For LLVM, I think the document is:
https://github.com/tensorflow/mlir/blob/master/g3doc/ConversionToLLVMDialect.md
It has some examples and some tips, but it needs more love. In the
end, we'd need three things: reasonable documents, at least one
implementation in-tree and good testing coverage.
As with any new technology that we introduce to LLVM, these things can
build up with time. Unlike them, however, MLIR is an existing project
with its own responsibilities. There will be a period of instability
for both projects as they are merged.
So, as long as we understand the costs and are willing to pay them,
each of the three things can come at "reasonable" time, after the
merge.
I'm assuming that the general approach is to, in case of conflict,
value LLVM's stability more than MLIR's during the transition. I'm
also assuming the transition will not take longer than one release
period (1/2 year).
I'm uncertain how the other projects that use MLIR will interact with
the LLVM community, but I'm also assuming that's a given, by wanting
to merge the two communities.
IE, I hope we don't get to the point where other users of MLIR want to
take a radically different direction as LLVM does, which would start a
conflict.
I don't think anyone here wants that, but it's good to be aware that
it could happen, and prepare for it.
cheers,
--renato
Alex Zinenko via llvm-dev
On Wed, 11 Sep 2019 at 23:04, Mehdi Amini <ami...@google.com> wrote:
> The MLIR LangRef corresponds to the former part only, because this is what is common to all dialects. On the other hand, each dialect will need to provide its own LangRef equivalent (for example I linked to the Affine dialect doc before).
For LLVM, I think the document is:
https://github.com/tensorflow/mlir/blob/master/g3doc/ConversionToLLVMDialect.md
Chris Lattner via llvm-dev
> On Sep 12, 2019, at 3:03 AM, Renato Golin <reng...@gmail.com> wrote:
>
> As with any new technology that we introduce to LLVM, these things can
> build up with time. Unlike them, however, MLIR is an existing project
> with its own responsibilities. There will be a period of instability
> for both projects as they are merged.
Yep, you’re right that MLIR is still early and we can build these things up over time.
One point of clarification though: MLIR was and has always been built with the idea that it would go to LLVM. This is why it has always followed the coding style, development practices, etc. The ‘instability’ that I expect is more about the GitHub infra changing (monorepo etc) than the code itself.
To put it another way, MLIR was built the way Clang was (both Clang and MLIR were a started as private projects that was eventually contributed to LLVM, with full revision control history). In contrast, MLIR isn’t being built the way LLDB was, which was a project that built up over time and then was later decided to move to LLVM.
-Chris
David Greene via llvm-dev
> The MLIR LangRef corresponds to the former part only, because this is what
> is common to all dialects. On the other hand, each dialect will need to
> provide its own LangRef equivalent (for example I linked to the Affine
> dialect doc before).
>
> Does it make sense?
Yeah, it makes perfect sense. I think maybe reading the document I got
caught up in the BNF grammar -- it's a bit distracting. I've not seen a
similar BNF specification for LLVM IR. It may exist somewhere, but BNF
isn't part of any LLVM document I've read. :) Maybe the grammar bits
could be factored out into a formal specification and LangRef could be a
little more informal.
Just suggestions, obviously.
Mehdi AMINI via llvm-dev
Mehdi Amini <ami...@google.com> writes:
> The MLIR LangRef corresponds to the former part only, because this is what
> is common to all dialects. On the other hand, each dialect will need to
> provide its own LangRef equivalent (for example I linked to the Affine
> dialect doc before).
>
> Does it make sense?
Yeah, it makes perfect sense. I think maybe reading the document I got
caught up in the BNF grammar -- it's a bit distracting. I've not seen a
similar BNF specification for LLVM IR. It may exist somewhere, but BNF
isn't part of any LLVM document I've read. :) Maybe the grammar bits
could be factored out into a formal specification and LangRef could be a
little more informal.
Just suggestions, obviously.
David Greene via llvm-dev
> Another things that I just remember now about documentation is that we
> don't expect dialects to write a "LangRef" that describe each
> operation. Instead we use a table-driven approach for defining
> operation
> and we generate both the C++ classes and the documentation from there
> (this helps keeping documentation up-to-date as well!).
>
> From a local build directory of MLIR, you can try:
>
> {BUILD}/bin/mlir-tblgen -gen-op-doc {SRC}/Dialect/StandardOps/Ops.td -I
> {SRC}/include/ > std.md
>
> (try --gen-op-defs and --gen-op-decls for the C++ code)
>
> I pushed these here for your convenience:
> https://github.com/joker-eph/mlir-docs/
Very nice!
> Of course we need to improve the content in general, but I expect the
> incentive to do so to grow assuming we can get a space like
> http://llvm.org/mlir ; at which point we could organize the MLIR overall
> online doc structure to include these generated file continuously.
That would be wonderful. Thanks for engaging on this!
Tanya Lattner via llvm-dev
Chris Lattner via llvm-dev
Tatiana Shpeisman via llvm-dev
This is great news! Congratulations everyone!On Mon, Oct 7, 2019 at 9:14 PM Tatiana Shpeisman <shpe...@google.com> wrote:Congratulations, everybody!