Adapting and open-sourcing PGI's Fortran frontend for LLVM

[Hal Finkel]

Hi everyone,

I have some very good news for everyone interested a production-quality Fortran frontend for LLVM:

The U.S. Department of Energy’s National Nuclear Security Administration and its three national labs have reached an agreement with NVIDIA's PGI division to adapt and open-source PGI's Fortran frontend, and associated Fortran runtime library, for contribution to the LLVM project. PGI is now working on isolating the necessary frontend components from their existing compiler code base and adapting those components to produce LLVM IR for consumption by our existing infrastructure.

We're expecting an initial source-code release from PGI by late 2016, and while that's still far away, PGI's developers will be working closely with the community to coordinate development efforts by others interested in Fortran support (e.g. on ABI support code for other platforms). Even before source code is available, PGI is going to keep the community updated on their progress. After source code is released, PGI will continue development work in coordination with the community to implement additional Fortran and OpenMP features not currently supported by PGI Fortran. The frontend will use LLVM's infrastructure for regression testing, and PGI will be developing regression tests for these new features in accordance with our existing practice.

I realize this is somewhat unconventional, but I'm confident that this approach will allow the LLVM community to capture the considerable Fortran expertise of PGI's team and provide our project with a production-quality LLVM frontend in a relatively short amount of time.

For the official announcement, please see:

https://www.llnl.gov/news/nnsa-national-labs-team-nvidia-develop-open-source-fortran-compiler-technology

-Hal

--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

[C Bergström]

I won't arrive in time to ask @SC15 on Sunday, but some questions

Will this include their CAF runtime? What if any vendor extensions
will be supported? What about directives support like OpenACC/OpenMP?
Is the code itself written in C/C++ or something else? Which Fortran
standards will it support? Will they be open sourcing corresponding
tests?

> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Hal Finkel]

----- Original Message -----
> From: "C Bergström" <cberg...@pathscale.com>
> To: "Hal Finkel" <hfi...@anl.gov>
> Cc: "LLVM Dev" <llvm...@lists.llvm.org>, "flang-dev" <flan...@googlegroups.com>, "Rob Neely" <nee...@llnl.gov>,
> "douglas miles (PGI)" <dougla...@pgroup.com>
> Sent: Friday, November 13, 2015 4:32:16 PM
> Subject: Re: [llvm-dev] Adapting and open-sourcing PGI's Fortran frontend for LLVM
>
> I won't arrive in time to ask @SC15 on Sunday, but some questions
>
> Will this include their CAF runtime? What if any vendor extensions
> will be supported? What about directives support like OpenACC/OpenMP?
> Is the code itself written in C/C++ or something else? Which Fortran
> standards will it support? Will they be open sourcing corresponding
> tests?

I'm going to defer to the PGI folks on answering these questions for now (except to note that, as stated below, there will be OpenMP support).

-Hal

[Hal Finkel]

----- Original Message -----
> From: "Alex Bradbury" <a...@asbradbury.org>
> To: "Hal Finkel" <hfi...@anl.gov>
> Cc: "LLVM Dev" <llvm...@lists.llvm.org>, "flang-dev" <flan...@googlegroups.com>, "Rob Neely" <nee...@llnl.gov>,
> "douglas miles (PGI)" <dougla...@pgroup.com>
> Sent: Saturday, November 14, 2015 1:41:08 AM
> Subject: Re: [llvm-dev] Adapting and open-sourcing PGI's Fortran frontend for LLVM
>

> On 13 November 2015 at 22:22, Hal Finkel via llvm-dev

> <llvm...@lists.llvm.org> wrote:
> > Hi everyone,
> >
> > I have some very good news for everyone interested a
> > production-quality Fortran frontend for LLVM:
> >
> > The U.S. Department of Energy’s National Nuclear Security
> > Administration and its three national labs have reached an
> > agreement with NVIDIA's PGI division to adapt and open-source
> > PGI's Fortran frontend, and associated Fortran runtime library,
> > for contribution to the LLVM project. PGI is now working on
> > isolating the necessary frontend components from their existing
> > compiler code base and adapting those components to produce LLVM
> > IR for consumption by our existing infrastructure.
> >
> > We're expecting an initial source-code release from PGI by late
> > 2016, and while that's still far away, PGI's developers will be
> > working closely with the community to coordinate development
> > efforts by others interested in Fortran support (e.g. on ABI
> > support code for other platforms). Even before source code is
> > available, PGI is going to keep the community updated on their
> > progress. After source code is released, PGI will continue
> > development work in coordination with the community to implement
> > additional Fortran and OpenMP features not currently supported by
> > PGI Fortran. The frontend will use LLVM's infrastructure for
> > regression testing, and PGI will be developing regression tests
> > for these new features in accordance with our existing practice.
>

> What will happen to flang development efforts?

This is obviously a somewhat-tricky question. There is an existing Flang code base and a few people have been working on it. Furthermore, it is still the only one we'll have for the next year. My hope is that, with the participation of the PGI developers, we'll be able to structure our Flang efforts on pieces that we'll be able to pick up and move to the code that PGI is releasing once that happens. Between development on the Driver, ABI support code, and testing and other infrastructure, the community can make substantial progress in 2016. By the end of 2016, we'll combine these pieces with PGI's code, and have a production-quality LLVM Fortran compiler representing the totality of this work.

-Hal

>
> Alex

[C Bergström]

On Sat, Nov 14, 2015 at 2:41 PM, Alex Bradbury via llvm-dev
<llvm...@lists.llvm.org> wrote:
>
> What will happen to flang development efforts?

I hope Hal is correct, but more likely he's just giving a tactful and
politically correct answer.

Reality is more likely that flang will have better code and design.
We'll end up with a Fortran front-end that mirrors the current quality
and capability as PGI. Frankly and no offence meant, but I would have
been much happily if they had just sponsored development on flang and
or worked with NAG (who has a much stronger reputation for actual
Fortran conformance - I've never tested PGI tbh)

Keep in mind that PGI's Fortran fe is likely c and has been around for
a while.. (Since the 90's?) The PGI engineers are welcomed to placate
my curiosity and correct any wrong assumptions. I hope like heck that
we don't end up with actual Fortran code (in the FE or runtime). That
would introduce a circular build dependency which I can assure you is
a major pita.

Other curiosities - will the module format be compatible with PGI
compiler? What about F2K8 and future standards? Arrays? Will they
(when?) publicly document how they are mapping things in Fortran to
llvm ir or we'll just end up with a code drop? (I personally know of a
few challenges with mapping Fortran to llvm ir)

Will we have to do a public information request or how much is PGI
being paid for this..

[John Leidel]

+1 to Hal's note regarding.  I'm personally in full support of PGI efforts (I am not an employee of NVIDIA or PGI).  I believe this is a great step for the future of Fortran as associated with LLVM.  I hope to start the discussions with the PGI folks this week at Supercomputing.  It should be an interesting year in development!

--
You received this message because you are subscribed to the Google Groups "flang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flang-dev+...@googlegroups.com.
To post to this group, send email to flan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flang-dev/15147591.1205.1447491564549.JavaMail.javamailuser%40localhost.

For more options, visit https://groups.google.com/d/optout.

[Carlo Bertolli]

Hi

Just wanted to share with everybody my support to PGI efforts in Flang. I was happy to learn that we are going to fast-forward Flang to PGI's level!

Thanks!

-- Carlo

To view this discussion on the web visit https://groups.google.com/d/msgid/flang-dev/CAC6GRO-LFy24T_ZVghciPtdU03aoc4XtviJY%3DqyN%3Dm4inoTqyw%40mail.gmail.com.

[Jeff Hammond]

With respect to the runtime requirements of Fortran 2008 coarrays, it would be nice if they switched to OpenCoarrays (http://opencoarrays.org/).  OpenCoarrays currently supports GASNet and MPI-3, which provides very nice portability.  It would be relatively straightforward for folks to port it to OFI (the next-generation Open Fabrics Interface - http://ofiwg.github.io/libfabric/) or UCX (has some connection to OpenPOWER).

Jeff

[Hal Finkel]

[-llvm-dev]

Hi again,

I had a productive conversation with Bob Scollard (PGI), Chandler, and some others in Austin last week that I'd like to briefly summarize for everyone else.

PGI will be releasing the necessary components to form an Fortran frontend sometime over the next year, and as a community, we need to decide how those components will relate to the Flang project. As we hope that Flang will become part of the LLVM project, this means deciding how these components relate to the LLVM project.

PGI's frontend is currently written in ANSI C, and they have no immediate plans to change that. They also currently plan on using a decoupled method of generating LLVM IR, specifically by printing textual IR for use by other LLVM components, and thus not linking directly to any of LLVM itself. Their parser is generated by a custom parser generator (for which they'll release source), and is not recursive descent. I'm not clear on the structure of the components after parsing. The runtime library is mostly written in C, with some pieces written in Fortran. Bob, if I've gotten anything wrong here, please correct me.

As Chandler was quick to point out, a lot of the power of Clang's design, especially in terms of generating good diagnostic messages, comes from its design as a recursive descent parser. The faithfulness of Clang's AST is also a strong enabler for many things (analysis and transformation tools, for example). The Flang community might not, therefore, want a Fortran parser constructed in a different way.

Moreover, the community, especially the larger LLVM community, will likely insist on a project built in C++ using LLVM libraries and coding standards, for inclusion in LLVM itself. As a result, aside perhaps from the runtime library, we'll need to consider carefully which pieces Flang we can borrow from the code that PGI will contribute more-or-less as-is, which pieces we'll want to take with major refactoring, and which we'll want to rewrite (perhaps treating PGI's code as the reference implementation).

That having been said, I feel comfortable based on this conversation, recommending that we continue developing the existing Flang codebase while we await code from PGI. We'll almost certainly want a differently-structured parser, at least, in the long term. That having been said, Bob stressed to me that PGI is interested in working in coordination with the wider community, and so they'll be part of the Flang community as we make our short-term and longer-term plans.

Thanks again,
Hal

> --
> You received this message because you are subscribed to the Google
> Groups "flang-dev" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to flang-dev+...@googlegroups.com.
> To post to this group, send email to flan...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/flang-dev/28947050.754.1447453317969.JavaMail.javamailuser%40localhost.

> For more options, visit https://groups.google.com/d/optout.
>