[llvm-dev] Pass reduction with new PM

[Markus Lavin via llvm-dev]

Hi,

 

Right now I am trying to analyze a bug that appears with new PM pipeline as follows

 

$ opt -enable-new-pm=1 -disable-basic-aa -tbaa -O1 -o /dev/null reduced.ll

 

the crashing pass is LICM but simply exporting IR before that pass and then running just that pass does not seem to capture enough state to reproduce the problem.

 

So here one would either want bugpoint to help automatically reduce the pass list or simply use -debug-pass=Arguments and do it manually but as I understand it neither of those options work with the new PM.

 

This has already been brought up in [llvm-dev] How to get from "opt -O3" to "opt <a-single-pass>" with the new PM?

Has there been any progress since then?

 

br

Markus

 

[David Blaikie via llvm-dev]

+Arthur for more NPM context

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Arthur Eubanks via llvm-dev]

There still is nothing like bugpoint's pass list reduce for the new PM.

The issue is that ideally we'd be able to map a pass pipeline to the string version of it. But we can't really do that right now. We could attempt to do that, but a generated argument list won't be exact since passes added via code (e.g. PassManagerBuilder/PassBuilder.cpp) often have parameters that can't be represented via opt arguments. Maybe getting an approximate textual representation of the pipeline is good enough for some cases? But that doesn't seem ideal.

We could also go down the route of making all pipelines representable via text, but that's a non-trivial project.

Note that even the legacy PM has this problem, and I've heard that this has bitten people in the past.

Perhaps as a short term hack, we could get an approximation of the pipeline by hacking PrintPassInstrumentation to print the textual name of each pass that's run via `PIC->getPassNameForClassName(PassID)` (like PrintIRInstrumentation). Then run `opt -debug-pass-manager with that hacked up version of PrintPassInstrumentation. We'd also need another version of -debug-pass-manager that prints adaptors/pass managers but doesn't print analyses. We'd run `opt` over a file with one function that contains one loop, e.g. the IR in new-pm-defaults.ll. Then cleanup the output of -debug-pass-manager with proper pass nesting.

Or of course you could attempt to manually recreate the proper textual representation by looking at PassBuilder.cpp.

Then we'd need something that reduces the textual pass pipeline.

Let me know if you'd like more details.

On Fri, Jul 30, 2021 at 12:25 AM Markus Lavin via llvm-dev <llvm...@lists.llvm.org> wrote:

_______________________________________________

[Markus Lavin via llvm-dev]

I firmly believe that something that almost works is a lot better than nothing that certainly won’t work at all so I suggest we go for the short term hack as you described.

 

I can try giving it a go but please provide all the details you have on that option.

 

So as far as I can see getting pass names using `PIC->getPassNameForClassName(PassID)` seems straight forward. Looks like we can get printouts when a PassManager and PassAdopter starts but I cannot see any callbacks that would be called when it is finished so maybe that needs to be added to be able to get the nesting right.

 

I think that reduction of the passes string can be handled by an external tool and for starters I would be happy to do it manually.

 

br

-Markus

[Björn Pettersson A via llvm-dev]

HI Markus.

 

I made some attempts to print both class name and pass name earlier this year.

 

One problem was that the getPassNameForClassName data base didn’t use a 1-1 mapping, so many “pass names” where mapped to the same “class name”.

I made some patches for that, but there might still be some 1-n mappings left (for example https://reviews.llvm.org/D105007 was never accepted so it remains to be dealt with).

 

Below are parts of the patch I was working on to print the pass names.

Might notice that we need replace the logic in shouldPopulateClassToPassNames in some better way and not just use “true” to always populate the database(or can we always do that?).

 

And there is still a limitation that for passes with params we still won’t print the params (had perhaps been nice to at least indicate that by printing “pass-name<…>” to show that the pass use params but that they aren’t shown for passes with params).

 

diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h b/llvm/include/llvm/Passes/StandardInstrumentations.h

index 9011c52f20c1..df4004b10525 100644

--- a/llvm/include/llvm/Passes/StandardInstrumentations.h

+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h

@@ -102,6 +102,7 @@ private:

   bool Enabled;

   PrintPassOptions Opts;

   int Indent = 0;

+  PassInstrumentationCallbacks *PIC;

};

 class PreservedCFGCheckerInstrumentation {

diff --git a/llvm/lib/Passes/PassBuilder.cpp b/llvm/lib/Passes/PassBuilder.cpp

index 54c3289f538f..7fec0e63cb88 100644

--- a/llvm/lib/Passes/PassBuilder.cpp

+++ b/llvm/lib/Passes/PassBuilder.cpp

@@ -445,7 +445,7 @@ AnalysisKey NoOpLoopAnalysis::Key;

/// it. This should be updated if new pass instrumentation wants to use the map.

/// We currently only use this for --print-before/after.

bool shouldPopulateClassToPassNames() {

-  return !printBeforePasses().empty() || !printAfterPasses().empty();

+  return !printBeforePasses().empty() || !printAfterPasses().empty() || true;

}

 } // namespace

 

diff --git a/llvm/lib/Passes/StandardInstrumentations.cpp b/llvm/lib/Passes/StandardInstrumentations.cpp

index a03e0d4b597e..3745c922579c 100644

--- a/llvm/lib/Passes/StandardInstrumentations.cpp

+++ b/llvm/lib/Passes/StandardInstrumentations.cpp

@@ -870,6 +871,8 @@ raw_ostream &PrintPassInstrumentation::print() {

 void PrintPassInstrumentation::registerCallbacks(

     PassInstrumentationCallbacks &PIC) {

+  this->PIC = &PIC;

+

   if (!Enabled)

     return;

@@ -884,7 +887,9 @@ void PrintPassInstrumentation::registerCallbacks(

         assert(!isSpecialPass(PassID, SpecialPasses) &&

                "Unexpectedly skipping special pass");

-        print() << "Skipping pass: " << PassID << " on " << getIRName(IR)

+        print() << "Skipping pass: " << PassID

+                << " (" << this->PIC->getPassNameForClassName(PassID) << ")"

+                << " on " << getIRName(IR)

                 << "\n";

       });

   PIC.registerBeforeNonSkippedPassCallback([this, SpecialPasses](

@@ -892,7 +897,9 @@ void PrintPassInstrumentation::registerCallbacks(

     if (isSpecialPass(PassID, SpecialPasses))

       return;

-    print() << "Running pass: " << PassID << " on " << getIRName(IR) << "\n";

+    print() << "Running pass: " << PassID

+            << " (" << this->PIC->getPassNameForClassName(PassID) << ")"

+            << " on " << getIRName(IR) << "\n";

     Indent += 2;

   });

   PIC.registerAfterPassCallback(