"deprecating" and 'removing" are different.
For 14.0.0, do you plan that -DLLVM_ENABLE_NEW_PASS_MANAGER=off will
give a warning or will be completely unsupported?
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Since deprecating something is giving warning that it’s going to be removed, I don’t think it’s fair to deprecate and remove in one step. If legacy pass manager isn’t currently formally deprecated (as in, loudly complains when you try to use it), then I’d personally like to see it deprecated for a release before any steps to completely remove it are taken.
From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of
Arthur Eubanks via llvm-dev
Sent: Tuesday, August 24, 2021 12:10 PM
To: Fangrui Song <mas...@google.com>
Cc: llvm-dev <llvm...@lists.llvm.org>
Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline
WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
I just wanted to say the same thing:)
I don't mind that we install a warning for -DLLVM_ENABLE_NEW_PASS_MANAGER=off
in the release/13.x branch so that the signal is clearer to downstream users.
> From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of Arthur Eubanks via llvm-dev
> Sent: Tuesday, August 24, 2021 12:10 PM
> To: Fangrui Song <mas...@google.com>
> Cc: llvm-dev <llvm...@lists.llvm.org>
> Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline
>
>
>
> WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>
> I probably should have said "deprecating and removing".
>
>
>
> My intention was to remove it completely. -DLLVM_ENABLE_NEW_PASS_MANAGER wouldn't do anything and we'd remove the -flegacy-pass-manaager/-fexperimental-new-pass-manager flags, as well as the corresponding lld flags.
>
>
>
> On Tue, Aug 24, 2021 at 12:02 PM Fangrui Song <mas...@google.com> wrote:
>
> On 2021-08-24, Arthur Eubanks via llvm-dev wrote:
> >The new pass manager has been on by default since the 13 branch. Now that
> >we've branched for 14, I'd like to start the process of deprecating and
> >removing legacy pass manager support for the optimization pipeline. This
> >includes clang, opt, and lld LTO support.
> >
> >Note that this doesn't apply to the codegen pipeline since there's no new
> >pass manager support for that yet.
> >
> >Are there any objections?
>
> "deprecating" and 'removing" are different.
>
> For 14.0.0, do you plan that -DLLVM_ENABLE_NEW_PASS_MANAGER=off will
> give a warning or will be completely unsupported?
--
宋方睿
Downstreams could be re-enabling the legacy pass manager in CMake. It’s still there, and still supported. If a downstream has made non-trivial changes to the pass pipeline for legacy pass manager, it’s going to be a ton of work to ensure performance parity when enabling new pass manager.
I think, if LLVM is built with LLVM_ENABLE_NEW_PASS_MANAGER set to OFF, then you should get a big scary warning at CMake configure time, but clang/opt should not complain. If LLVM is built with DLLVM_ENABLE_NEW_PASS_MANAGER set to ON, but the flag is passed to clang/opt to build using the legacy pass manager, you should get a warning from clang/opt.
In my opinion, if these deprecation warnings make it in for LLVM 13, then it’s fine to begin removing LPM in LLVM 14.
thanks,
Chris Tetreault
Let me clarify that when I say “it’s fine to begin removing LPM in LLVM 14”, I mean that it’s fine to begin removing LPM once LLVM 14 is released, and the version string in main is set to 15.
While I think such a deprecation and removal policy may be fine for many non-trivial things, asking
this may be too much for the pass manager. With a grain of salt, "downstreams are on their own." New
PM migration and legacy PM removal has been repeatedly forewarned. I think llvm-project has done
above and beyond what it was expected. The flip in git was done in February.
After 13.0.0 is released for, say 2 months, which additional attests its robustness, I think
removing non-codegen legacy PM pieces should be fine.
>From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of Chris Tetreault via llvm-dev
>Sent: Tuesday, August 24, 2021 3:39 PM
>To: Arthur Eubanks <aeub...@google.com>; Fāng-ruì Sòng <mas...@google.com>
>Cc: llvm...@lists.llvm.org
>Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline
>
>
>WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>Downstreams could be re-enabling the legacy pass manager in CMake. It’s still there, and still supported. If a downstream has made non-trivial changes to the pass pipeline for legacy pass manager, it’s going to be a ton of work to ensure performance parity when enabling new pass manager.
>
>I think, if LLVM is built with LLVM_ENABLE_NEW_PASS_MANAGER set to OFF, then you should get a big scary warning at CMake configure time, but clang/opt should not complain. If LLVM is built with DLLVM_ENABLE_NEW_PASS_MANAGER set to ON, but the flag is passed to clang/opt to build using the legacy pass manager, you should get a warning from clang/opt.
>
>In my opinion, if these deprecation warnings make it in for LLVM 13, then it’s fine to begin removing LPM in LLVM 14.
>
>thanks,
> Chris Tetreault
>
>From: Arthur Eubanks <aeub...@google.com<mailto:aeub...@google.com>>
>Sent: Tuesday, August 24, 2021 1:47 PM
>To: Fāng-ruì Sòng <mas...@google.com<mailto:mas...@google.com>>
>Cc: Chris Tetreault <ctet...@quicinc.com<mailto:ctet...@quicinc.com>>; llvm...@lists.llvm.org<mailto:llvm...@lists.llvm.org>; Tom Stellard <tste...@redhat.com<mailto:tste...@redhat.com>>
>Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline
>
>
>WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>If nobody is setting -DLLVM_ENABLE_NEW_PASS_MANAGER=off then I'm not sure that we need to go through the whole deprecation dance. Is there anybody doing that?
>
>But if we do the deprecation dance, should it be a CMake warning or a clang warning? I'm not sure people will notice a CMake warning.
>And would putting that in the 13.x branch be good enough to start removing after 14.x?
>
>On Tue, Aug 24, 2021 at 12:26 PM Fāng-ruì Sòng <mas...@google.com<mailto:mas...@google.com>> wrote:
>On Tue, Aug 24, 2021 at 12:21 PM Chris Tetreault <ctet...@quicinc.com<mailto:ctet...@quicinc.com>> wrote:
>>
>> Since deprecating something is giving warning that it’s going to be removed, I don’t think it’s fair to deprecate and remove in one step. If legacy pass manager isn’t currently formally deprecated (as in, loudly complains when you try to use it), then I’d personally like to see it deprecated for a release before any steps to completely remove it are taken.
>
>I just wanted to say the same thing:)
>
>I don't mind that we install a warning for -DLLVM_ENABLE_NEW_PASS_MANAGER=off
>in the release/13.x branch so that the signal is clearer to downstream users.
>
>> From: llvm-dev <llvm-dev...@lists.llvm.org<mailto:llvm-dev...@lists.llvm.org>> On Behalf Of Arthur Eubanks via llvm-dev
>> Sent: Tuesday, August 24, 2021 12:10 PM
>> To: Fangrui Song <mas...@google.com<mailto:mas...@google.com>>
>> Cc: llvm-dev <llvm...@lists.llvm.org<mailto:llvm...@lists.llvm.org>>
>> Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline
>>
>>
>>
>> WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>>
>> I probably should have said "deprecating and removing".
>>
>>
>>
>> My intention was to remove it completely. -DLLVM_ENABLE_NEW_PASS_MANAGER wouldn't do anything and we'd remove the -flegacy-pass-manaager/-fexperimental-new-pass-manager flags, as well as the corresponding lld flags.
>>
>>
>>
>> On Tue, Aug 24, 2021 at 12:02 PM Fangrui Song <mas...@google.com<mailto:mas...@google.com>> wrote:
>>
>> On 2021-08-24, Arthur Eubanks via llvm-dev wrote:
>> >The new pass manager has been on by default since the 13 branch. Now that
>> >we've branched for 14, I'd like to start the process of deprecating and
>> >removing legacy pass manager support for the optimization pipeline. This
>> >includes clang, opt, and lld LTO support.
>> >
>> >Note that this doesn't apply to the codegen pipeline since there's no new
>> >pass manager support for that yet.
>> >
>> >Are there any objections?
>>
>> "deprecating" and 'removing" are different.
>>
>> For 14.0.0, do you plan that -DLLVM_ENABLE_NEW_PASS_MANAGER=off will
>> give a warning or will be completely unsupported?
>
>
>
>--
>宋方睿
On 2021-08-24, Chris Tetreault wrote:
>Let me clarify that when I say “it’s fine to begin removing LPM in LLVM 14”, I mean that it’s fine to begin removing LPM once LLVM 14 is released, and the version string in main is set to 15.
While I think such a deprecation and removal policy may be fine for many non-trivial things, asking
this may be too much for the pass manager.
I'd vote for immediate removal. I don't have much sympathy for downstream consumers who haven't moved. This effort has been underway for literal years. Many - though not by any means all - downstream projects moved *before* upstream finally switched. Let's put a nail in this coffin, and remove code aggressively.
Supporting both has serious ongoing costs. As a real example, I have twice spent time in the last two weeks tracking down some odd quirk of the unrolling implementation to find it supports some quirk of the legacy pass. That slows down development, compromises quality, and is generally a "bad thing".
We might want to wait a couple of weeks/months to ensure
stability, but we should only consider the needs to the upstream
project itself when doing so. Giving downstream projects time to
react should be an explicit non-goal.
Philip
p.s. I don't expect this to be the actual decision reached, but
since I only see opinions down-thread arguing for migration
windows, I wanted to make a point of sharing the opposite
opinion. Fair warning, I probably won't reply to this thread
further. I don't have sufficient interest in the topic to make it
worthwhile.
I think that’s a sufficiently obnoxious warning. I still strongly prefer that no removals of functionality come until we branch for LLVM 14, but I think this will do for a notice of deprecation.
Thanks,
Chris Tetreault
From: Arthur Eubanks <aeub...@google.com>
Sent: Friday, August 27, 2021 12:42 PM
To: Chris Tetreault <ctet...@quicinc.com>
Cc: llvm-dev <llvm...@lists.llvm.org>
Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline
WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
Are https://reviews.llvm.org/D108789 and https://reviews.llvm.org/D108775 sufficient if we cherrypick them into 13?
This is a good point. I’ve stated my preferences, but I think we can all agree that it would be good to settle on a concrete timeline of what’s getting removed, and when.
Thanks,
Chris Tetreault
From: Mehdi AMINI <joke...@gmail.com>
Sent: Friday, August 27, 2021 3:54 PM
To: Arthur Eubanks <aeub...@google.com>
The previous disabling mechanism
-DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=off is ignored in main and
release/13.x.
If a user switches (-DLLVM_ENABLE_NEW_PASS_MANAGER=off), they should
notice the warning.
So I think we don't need something like
-DLLVM_FORCE_USE_OLD_TOOLCHAIN=on which was done in 2019 to bump the
toolchain requirement.
--
宋方睿
For the record, I am a downstream user who is speaking up.
Migrating a customized downstream from legacy pass manager to new pass manager involves more than just porting a few passes. Several passes (such as the inliner) were rewritten from scratch. Effort was made to make the new versions of these rewritten passes match the original, but the interaction of the whole pass pipeline can be very fickle. What was determined to be good enough upstream might not be quite right in the face of downstream modifications. And if these rewritten passes were customized, then these customizations will be lost. Additionally, the construction of the pass pipeline has been fine tuned over years by the community, and by downstream users to produce good results, and mitigating regressions caused by switching to the new pass manager takes time. It’s easy to miss changes, and this can result in spending a bunch of time trying to figure out why some workload had a 3% regression.
I deal with this stuff every day at work, so I feel the pain of maintaining two pass infrastructures. However, I don’t understand what the hurry is. Who is blocked by legacy pass manager remaining operational? If maintaining the legacy pass manager is slowing the velocity of development in upstream, I think it’s reasonable to put it on life support and only fix correctness issues and egregious perf regressions. Downstreams that are ready can stop supporting it, but I don’t see why we can’t wait 6 more months to begin deleting code in upstream. I feel like “one whole release” is a pretty standard minimum period of time to deprecate major functionality in a software project. Sure, it’s been a long time coming, but legacy pass manager was never deprecated. No date was ever given for its removal.
I think that it is reasonable to state that legacy pass manager will be removed in the release of LLVM 14, and once the version string of main is set to 15 it’s open season to immediately begin gutting it. Prior to that, only a minimal level of effort need be made to maintain performance of the legacy pass manager. (i.e., downstreams who care about it can fix the bugs themselves) In the meantime, we can migrate the lit tests to new pass manager. New code that is added should prioritize the performance of the pass pipeline constructed with the new pass manager over that of the legacy pass manager. New passes that are added need not be ported to the legacy pass manager. All I’m asking is that we not delete code until we’ve had a real deprecation period.
Thanks,
Chris Tetreault
I appreciate it. This will be super helpful for us!
> On 31 Aug 2021, at 23:34, Arthur Eubanks via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> Debugging new PM regressions is reasonable. I thought people were just taking too long to port passes. Given that argument I'm content to wait until the next branch.
>
> We can start migrating lit tests as you say.
FWIW I think it makes sense to delay removing legacy PM code by six months, start moving lit tests now and also not require to investigate/fix legacy PM-only issues.
But I think it would be good to spell out the decision somewhere; maybe add a note that legacy PM support is due to be removed after the 14.0 release in the release notes on the 13.x and main branches? IMO that could be considered sufficient notice and we won’t be back to discussing whether to go ahead with the removal in 6 months again.
Cheers,
Florian