[llvm-dev] FC : A MLIR+LLVM based Fortran front end

[Prashanth N R via llvm-dev]

Hi-

In August we made an announcement  of "FC: A new fortran front end"  [1]. At that time to get an end-to-end solution, we made FC to emit LLVM IR directly.  At present, we have upgraded FC to emit MLIR. Currently the language supported is close to Fortran-95. Apart from 400+ unit test cases, out framework passes two SPEC-2017 benchmarks successfully.  Currently we are cleaning up the code and plan to open source the same in a week or so.

Apart from the front end changes, we have defined a new high level dialect in MLIR for representing Fortran. We have defined a new Loop Nest Optimization framework along with one Loop Nest Optimization in it for the time being. Also we have supported 4 OpenMP constructs and way to express the same in MLIR framework. 

Entire code is in the spirit of LLVM. If the community is interested, we would like to work with the community and move forward. 

thanks,

-Prashanth

PS: We did try to use F18 for a while and hit the issues raised in [2], [3].

1. http://lists.llvm.org/pipermail/llvm-dev/2019-August/134620.html

2. http://lists.llvm.org/pipermail/llvm-dev/2020-January/138069.html

3. http://lists.llvm.org/pipermail/flang-dev/attachments/20191202/6e7c1e51/attachment-0001.pdf

OR http://lists.llvm.org/pipermail/flang-dev/2019-December/000089.html

[Finkel, Hal J. via llvm-dev]

Hi, Prashanth,

That's great news! It sounds like you've made a lot of progress, and I certainly hope that you can make your source available under the LLVM license and that we can all work together going forward.

 -Hal

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

[Chris Lattner via llvm-dev]

Agreed!  Is the code already available?  What are your plans for it, and are you interested in collaboration with the rest of the LLVM community?

-Chris

[Prashanth N R via llvm-dev]

Hi Hal-

 In a day or two we will open source the code. Bigger clean up is expected by the end of this week. Of course the license will be LLVM license.

thanks,

-Prashanth

[Prashanth N R via llvm-dev]

Hi Chris-

We are mostly cleaning up the code for open sourcing. Early code drop will happen in a day or two and will continue through this week. We would love to work with the rest of the community on this. 

thanks,

-Prashanth

[James Y Knight via llvm-dev]

Neat, another fortran compiler option.

Does anyone have a list/comparison of all the LLVM fortran compilers? I'm not really tracking this, since Fortran isn't really my area of expertise, but I've seen the following. Perhaps there are even more?

"Flang". The original of the name, I think? Abandoned.
https://github.com/llvm-flang/flang
"Fort" -- fork of the above flang, renamed. Seems active.
https://github.com/llvm-fortran/fort/

"Flag". Different from the other "flang". Active, but deprecated in favor of f18.

https://github.com/flang-compiler/flang

"f18". Rewrite-from-scratch (?) of adjacent "flang". Active, but not fully-functional yet (I think?).

https://github.com/flang-compiler/f18

"FC". Newly-developed, independent of the other compilers. Active. (But this source-code dates from the previous announcement, not yet updated with latest developments):
https://github.com/compiler-tree-technologies/fc

I don't know how any of these compare with each other, nor why we have so many. I have no opinion on which or whether any of these should be "blessed", nor do I really want to become educated enough on this topic to gain such an opinion personally.

But, given that there appear to be at least 3 fortran compilers under active development by different people, I would love to be reassured that such a comparison has been done, and that we are selecting one of them for good reasons. It'd be great if someone (or multiple someones) could post a summary/comparison of what these things all are, and why one is the right path forward vs another.

[Finkel, Hal J. via llvm-dev]

On 1/13/20 11:02 AM, James Y Knight wrote:

Neat, another fortran compiler option.

Does anyone have a list/comparison of all the LLVM fortran compilers? I'm not really tracking this, since Fortran isn't really my area of expertise, but I've seen the following. Perhaps there are even more?

"Flang". The original of the name, I think? Abandoned.
https://github.com/llvm-flang/flang
"Fort" -- fork of the above flang, renamed. Seems active.
https://github.com/llvm-fortran/fort/

"Flag". Different from the other "flang". Active, but deprecated in favor of f18.

https://github.com/flang-compiler/flang

"f18". Rewrite-from-scratch (?) of adjacent "flang". Active, but not fully-functional yet (I think?).

https://github.com/flang-compiler/f18

"FC". Newly-developed, independent of the other compilers. Active. (But this source-code dates from the previous announcement, not yet updated with latest developments):
https://github.com/compiler-tree-technologies/fc

I don't know how any of these compare with each other, nor why we have so many. I have no opinion on which or whether any of these should be "blessed", nor do I really want to become educated enough on this topic to gain such an opinion personally.

But, given that there appear to be at least 3 fortran compilers under active development by different people, I would love to be reassured that such a comparison has been done, and that we are selecting one of them for good reasons. It'd be great if someone (or multiple someones) could post a summary/comparison of what these things all are, and why one is the right path forward vs another.

I don't view what we're doing, or should be doing, as selecting one of them. We should have a Fortran frontend in the LLVM project. It should be called Flang. It should be developed by a broad subset of our LLVM community.

A decision was made that it would be reasonable to import from the f18 project to start this process, in part because the founding purpose of the f18 activity was to produce a modern Fortran compiler in modern C++ that would fit with the LLVM community. Its development process has been open, many of us have been involved, and so on. Very important, however, is that it's under active development and that it's far from done. I want the design and implementation of the Frontend to be an LLVM community effort, and the best way for that to happen is for the process to happen under the umbrella of the LLVM project. Critically, this is what those involved the current f18 development actively desire.

I can't say all of the same about the other projects. However, I think that it is important to make clear that LLVM's Flang project is a community project and everyone who wishes to be a part of that community is welcome. As I see it, we now have a significant pool of people with LLVM/Fortran development experience who can, if they choose, contribute to our effort, and I actively encourage them to do so. That means that we might end up doing some things more-or-less like f18 does them now, and we might end up doing other things more-or-less like FC does them now, and so on. It all depends on who contributes and what feedback they receive from everyone else.

Thanks again,

Hal

[Prashanth N R via llvm-dev]

Hi-

The code has been open sourced at https://github.com/compiler-tree-technologies. There are two repositories.

1. The Fortran front end, aka FC, at https://github.com/compiler-tree-technologies/fc.

2. Associated LLVM changes including MLIR at https://github.com/compiler-tree-technologies/llvm-project.

Build instructions are given at the website. There is a document at https://github.com/compiler-tree-technologies/fc/blob/master/docs/FC_MLIR.pdf which gives low level implementation details. As the week progresses, we might clean up the code and upstream. In the meantime, if there are any questions, please do let me know.

thanks,

-Prashanth

On Mon, Jan 13, 2020 at 11:17 AM Chris Lattner <clat...@nondot.org> wrote:

[River Riddle via llvm-dev]

On Mon, Jan 13, 2020 at 1:13 PM Prashanth N R via llvm-dev <llvm...@lists.llvm.org> wrote:

Hi-

The code has been open sourced at https://github.com/compiler-tree-technologies. There are two repositories.

1. The Fortran front end, aka FC, at https://github.com/compiler-tree-technologies/fc.

2. Associated LLVM changes including MLIR at https://github.com/compiler-tree-technologies/llvm-project.

Hi,

(disclaimer: I can’t say much about the Fortran side, but I did look at some of the MLIR bits)

I took a look at the repos, and would love for you(+your team) to upstream any of the general MLIR bits you can. Feel free to add me as a reviewer. (I also went ahead and fixed one of your issues).

I look forward to collaborating with you more!

— River

--

Thank you,
River Riddle

[Prashanth N R via llvm-dev]

Hi River-

We are in the process of cleaning our code and we will be upstreaming our OpenMP and Loop optimization changes soon. It would be great to get your feedback on MLIR changes.

-Prashanth

[Prashanth N R via llvm-dev]

Hi Chris/Hal-

[2/3]

As agreed we are open sourcing implementation of 5 Openmp constructs i.e. omp.parallel, omp.single, omp.do, omp.master and omp.parallel_do. The latest code can be accessed at https://github.com/compiler-tree-technologies/fc .  Along with LLVM fork the repository can be accessed at https://github.com/compiler-tree-technologies/ .  Testing infrastructure has been upgraded to Lit framework. Openmp test cases can be accessed at https://github.com/compiler-tree-technologies/fc/tree/master/test/openmp .  Implementation of rest of the constructs will be open sourced in a later release.  Please give feedback on the implementation and report any issues you encounter. 

A detailed design note along with the issues considered will be sent shortly to the group for discussion. 

Thanks,

-Prashanth

PS: In the next release we will opensource implementation of some loop transforms like Loop Blocking, Loop Permute(generalized interchange) etc.

On Mon, Jan 13, 2020 at 11:17 AM Chris Lattner <clat...@nondot.org> wrote:

[Petr Penzin via llvm-dev]

Let me try to summarize what those are - hopefully this can be useful for somebody. Please this write up with a grain of salt (particularly on Fortran support), as I don't follow every project here very closely.

In somewhat chronological order:

*Flang (old)*
https://github.com/llvm-flang/flang
A deeply modified fork of Clang. Originally a GSoC project, but received contributions from numerous people after. Uses Clang-like AST, compiler diagnosics, tablegen, Lit for testing and so on. Recursive descent parser, similar to Clang's. Abandoned in favor of new Flang. Fortran support: F77, F90, and F03 (all partial)

*Flang (new)*
https://github.com/flang-compiler/flang
Unrelated effort under the same name. Was publicly announced as "derived from PGI Fortran compiler" [1], which is an existing (and relatively old) code base with a lot more features. Uses two stage compilation with two different executables (apart from the driver) doing the work. It emits some form of textual IR between stages and produces LLVM IR as text before handing it back to the rest of toolchain (via Clang). This project does not use any LLVM libraries directly. Its frontend employs its own parser generator [2]. If I remember correctly, it might have some issues with more complex features from Fortran 2003 or 2008 and newer standards, but is pretty solid at everything older. Despite its quirks it is the most used and most complete compiler on this list.

[1]: https://www.llnl.gov/news/nnsa-national-labs-team-nvidia-develop-open-source-fortran-compiler-technology
[2]: https://github.com/flang-compiler/flang/tree/master/tools/flang1/utils/prstab

*f18*
https://github.com/flang-compiler/f18
Ground-up rework of (new) Flang. The only entry on the list that does not perform full compilation yet. It was announced that it would eventually emit MLIR. It looks like parser is the only complete component as of today. As for internals, there is an evaluation [3] with some slides [4] posted on flang-dev list - looks like covering writing tools based on f18. A quick look at the sources shows some interesting use of overloaded '>>' operator [5] resembling monadic combination in functional programming, which is confirmed in one of the bug reports [6]. The rest of the same bug report is worth reading as well, as this programming style might lead to high resource usage while building the compiler.

[3]: http://lists.llvm.org/pipermail/flang-dev/2019-December/000089.html
[4]: http://lists.llvm.org/pipermail/flang-dev/attachments/20191202/6e7c1e51/attachment-0001.pdf
[5]: https://github.com/flang-compiler/f18/blob/master/lib/parser/stmt-parser.h#L24
[6]: https://github.com/flang-compiler/f18/issues/431#issuecomment-487664759

*Fort*
https://github.com/llvm-fortran/fort/
Renamed fork of old flang. Compatible with lasted LLVM, some improvements have been added. Probably the most Clang-like compiler on the list. Uses AST based on Clang's, tablegen, Lit, mostly same LLVM libraries which Clang uses. Just like its predecessor lacks driver. Supports parts of Fortan 77, Fortan 90, and Fortran 2003.

*FC*
https://github.com/compiler-tree-technologies/fc
Not related to any of the above, but seems to be in some ways similar to Clang/(old)Flang/Fort. Recursive descent parser; emits MLIR. Does not use tablegen, has a different way of throwing errors, has no builtin diagnostics, also no driver. Project has only seven commits to this date. Not clear on the Fortran standard support (has flags to support f77 and f95), the claim that it can compile a couple of larger application from SPEC, but I can't find which ones.

[Prashanth N R via llvm-dev]

Hi Petr-

1. We compile SPEC benchmarks bwaves and xchange currently. We are close to compiling fotonik at this point. 

2. We released the source as experimental for the community to take a look. Soon we will open source the entire history of development(we are working on the script to upload the history).

thanks,

-Prashanth

[Philip Reames via llvm-dev]

Thank you for the summary.  Don't know about anyone else, but this helped me a lot.  I'm only casually interested in the fortran efforts and had been getting lost as to which project was being referenced in various threads.

Philip

[Petr Penzin via llvm-dev]

Hi Prashanth,

Do those benchmarks run? They seem to be relatively short (bwaves [1], exchange [2]), but if the runs verify correctly that would be pretty good.

Do you plan on supporting any of the Fortran 2003 features?

-Petr

[1]: https://www.spec.org/cpu2017/Docs/runcpu-avoidance.html#pickone

[2]: https://www2.hawaii.edu/~nreed/ics606/papers/Metcalf06Sudoku.pdf

[Prashanth N R via llvm-dev]

Hi Petr-

Both the benchmarks run. We are currently working on fotonik and roms. 

We will support some of Fortran 2003 features soon.

thanks,

-Prashanth