Re: [llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup
Gaier, Bjoern via llvm-dev
Hi Lang,
Thank you for your answer! This helped me again a lot!! Also that ResourceTracker is a really neat feature! Looking forward to it! :3
I changed the title cause… there is another issue I have (sorry about that…)
I’m finally allowed to investigate the ORC JIT for integration into our system, which meant I got a few days to actually play around with it. However, another problem arise which breaks my concept. It is the never ending story of “cross references”. I have again two Modules, which are added to two different LLJIT instances, but they are referencing each other. In the past you suggested me to use the LookUp function of the ExecutionSession to get the addresses – so I wrote this:
bool ModuleLoader :: resolve()
{
auto &ES = this->jit->getExecutionSession();
SymbolLookupSet lookupSet;
this->undefinedReferences.clear();
for(const auto &element : this->symbolsOfInterrest)
{
lookupSet.add(element.second.name,llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
}
auto result = ES.lookup({{&jit->getMainJITDylib(), llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);
if(result)
{
for(const auto &element : *result)
{
const llvm::StringRef &name = (*element.first);
const size_t hash = calculateHash(name);
printf(">>>%s @ 0x%p\n", name.data(), element.second.getAddress());
this->symbolsOfInterrest[hash].adr = element.second.getAddress();
}
}
else
{
ES.reportError(result.takeError());
}
this->mtx.unlock();
return (this->undefinedReferences.size() == 0ull);
}
I also attached a reporter to the ES which will handle “llvm::orc::SymbolsNotFound”
by copying SymbolVector to the “undefinedReferences”. If I call this function and have every symbol resolved, then I can use the addresses to actually execute them. That is great! However, when I have an undefined Reference, things get strange… The first call
will trigger my “tryToGenerate” function but it will not be able to resolve a certain symbol. The reporter will be triggered and the “undefinedReferences” SymbolVector will have size 1.
When I call the function a second time however, the “tryToGenerate” function will not be called anymore, so my vector will be empty, the undefined reference is still not resolved, but I return with a true and crash my program. So even if I would have an address
for that one symbol in the second run, I would have no chance to tell “anyone” cause the “tryToGenerate” function was never called… Said function looks like that though:
llvm::Error ReferenceManager::UndefinedReferenceResolver :: tryToGenerate(llvm::orc::LookupKind K, llvm::orc::JITDylib &JD, llvm::orc::JITDylibLookupFlags JDLookupFlags, const llvm::orc::SymbolLookupSet &LookupSet)
{
llvm::orc::SymbolNameVector notFound;
llvm::orc::SymbolMap NewSymbols;
for(const auto &name : LookupSet)
{
printf("Generate!\n");
const uintptr_t adr = UndefinedReferenceResolver::lookup((*name.first).data());
if(adr)
{
NewSymbols[name.first] = llvm::JITEvaluatedSymbol(adr, llvm::JITSymbolFlags::Absolute);
}
else
{
notFound.push_back(name.first);
}
}
JD.define(absoluteSymbols(std::move(NewSymbols)));
return (notFound.size() == 0) ? llvm::Error::success() : llvm::make_error<llvm::orc::SymbolsNotFound>(std::move(notFound));
}
When I use “llvm::orc::JITDylibLookupFlags::MatchExportedSymbolsOnly” then I will get 1 undefined Reference in the first run, but a total of 9 in the second run, because every symbol I wanted to lookup was now an undefined reference.
Thank you for the help in advance!
Kind greetings
Björn
From: Lang Hames <lha...@gmail.com>
Sent: 23 September 2020 07:54
To: Gaier, Bjoern <Bjoern...@horiba.com>
Cc: LLVM Developers Mailing List <llvm...@lists.llvm.org>
Subject: Re: ORC JIT - Can modules independently managed with one LLJIT instance?
Hi Bjoern,
Sounds like you want the removable code feature that's under development in the orcv1-removal branch of https://github.com/lhames/llvm-project.git. I will be aiming to merge this work back into the mainline some time in the next couple of weeks.
I have not added a "removeJITDylib" method to ExecutionSession with this feature yet, but will try to design and publish that tomorrow. Then you should be able to do exactly what you want.
In the new system you can also perform fine grained removal: It is possible to track and remove individual modules from within a JITDylib. For an example of this see: https://github.com/lhames/llvm-project/blob/7ec9f8930f68760953a483157e010d0ff88285cd/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/toy.cpp#L1148, from lines 1148 to line 1161.
Also a related but different question:
When using llvm::parseIRFile I need to pass it a LLVMContext - should this be a unique one for every module I pass or can I use a global one?
It depends on whether you want to be able to compile concurrently in the JIT process. If you do want to compile concurrently then each module should get its own context. If you are happy to stick to single threaded compilation you can load all modules on the same context, potentially saving some memory.
-- Lang.
On Thu, Sep 17, 2020 at 11:59 PM Gaier, Bjoern <Bjoern...@horiba.com> wrote:
Hello everyone and Lang,
I have another design or "how-to" question about the ORC JIT. Sorry for having so many about them, to me this is a really complicated yet fascinating subject...
How would I design an ORC JIT with the following requirements?
At any time it should be possible to load a LLVM Module, every Module is independent and is not allowed to be linked with the other modules, every module can be removed at any time.
My first idea was to have an ORC JIT for every module I load, but then I wondered if I could use a single ORC JIT for it.
So, I would create an ORC JIT:
- using llvm::orc::LLJITBuilder
- configurating it
- adding a custom memory manager that requests the entire memory size
Then I would call "getMainJITDylib" and fill it with symbols that are valid for every module:
- adding printf, strlen, usw.
Now when I get a request to load a module:
- load module
- get symbols I want to look up
- create new DyLib and add module to that
Now I would do a lookup on that module, plus the main module to get symbol addresses and stuff
If a new module is added, it goes to a new DyLib as well and so on.
-----
However, now I want to delete a module again. Killing the memory is fairly easy with my custom memory manager, but I still have my DyLib of the now dead code... Can I get rid of it without dumping the others or the entire ORC JIT?
Or is there a different approach to do this? My goal with this is to speed up the setup phase for a module and also to reduce dynamic memory usage.
Also a related but different question:
When using llvm::parseIRFile I need to pass it a LLVMContext - should this be a unique one for every module I pass or can I use a global one?
Kind greetings and thank you all again,
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
Lang Hames via llvm-dev
Gaier, Bjoern via llvm-dev
Hey Lang,
I would be really happy to only have one LLJIT instance and using multiple JITDylibs. However… it seems like that I don’t know enough to use them. So I wonder…
- When I add Module A to JITDylib A and Module B to JITDylib B – where will those look for undefined symbols? Will Module A for example: will it only search itself and the MainDylib? Or would it also search in JITDylib B?
- If my current approach with using tryToGenerate and ES.lookUp is not correct, how would I do this then? Our cross references don’t have the exact symbol names where they want to be resolved to. Our modules are loaded into a tree structure so you can use the symbol names to navigate that tree. This is why I don’t want the LLJIT to do an automatic resolving of those symbols, because it can’t know about our structure. However, how could I achieve this then? Especially if it is not so straight forward to find the symbols that are not resolved yet.
For example: Module A might reference Planschi_test Module B was loaded with name “Planschi” and has a variable called test – so our old loader would then resolve that “Planschi_test”
reference with that address.
However those “pathes” can get really complex like navigating to a parent and such.
- Is there a way then, how I can lookup the addresses of the Module A, while still having undefined references, but I’m allowed to provide those at a later point?
About the multiple LLJIT approach:
When I used tryToGenerate for Module A I actually only looked in a hash map of Module B which I filled with their ExecutionSession::lookup function – so tryToGenerate didn’t called that function. However, like I said – a second call to the Module A lookup didn’t
triggered another call to tryToGenerate, giving me no chance to ask Module Bs hash map again :c
Thank you again ^o^
Lang Hames via llvm-dev
I would be really happy to only have one LLJIT instance and using multiple JITDylibs. However… it seems like that I don’t know enough to use them. So I wonder…
1. When I add Module A to JITDylib A and Module B to JITDylib B – where will those look for undefined symbols? Will Module A for example: will it only search itself and the MainDylib? Or would it also search in JITDylib B?
2. If my current approach with using tryToGenerate and ES.lookUp is not correct, how would I do this then? Our cross references don’t have the exact symbol names where they want to be resolved to. Our modules are loaded into a tree structure so you can use the symbol names to navigate that tree. This is why I don’t want the LLJIT to do an automatic resolving of those symbols, because it can’t know about our structure. However, how could I achieve this then? Especially if it is not so straight forward to find the symbols that are not resolved yet.
For example: Module A might reference Planschi_test Module B was loaded with name “Planschi” and has a variable called test – so our old loader would then resolve that “Planschi_test” reference with that address.
3. Is there a way then, how I can lookup the addresses of the Module A, while still having undefined references, but I’m allowed to provide those at a later point?
Gaier, Bjoern via llvm-dev
Hi Lang,
I think I start to get an idea how we could use LLVM in our system… so I try repeating what I understood:
> Searches follow the containing JITDylib's Link Order. E.g.
> auto &LibA = ExitOnErr(LLJIT.getExecutionSession().createJITDylib("LibA"));
> auto &LibB = ExitOnErr(LLJIT.getExecutionSession().createJITDylib("LibB"));
> LibA.addToLinkOrder(LibB); // Add LibB to LibA's search order.
That means – when I don’t add LibB to LibA, that this one will not be searched right?
However… when I add LibB to LibA and now I add LibA to LibC – will LibC also search LibB?
LibA.addToLinkOrder(LibB);
LibC.addToLinkOrder(LibA);
---------
> A. Rename variables/functions at the IR level or higher. E.g. Rename 'test' to 'Planschi_test' in IR. This is probably the simplest scheme if your use-case allows.
In my case this would mean… I rename “Planschi_test” to “test” and would add the “LibPlanschi” to my LinkOrder – however. What if my new Module also defines a variable “test”? Wouldn’t this cause trouble?
> B. Use a DefinitionGenerator and the re-exports utility: When you see an external reference to M_N you would create a re-export of N -> (JITDylibForM, N). This is ok, but you'll need to manage extra JITDylibs to ensure that no two JITDylibs end up containing a duplicate definition (e.g. 'test').
How would that work? Wouldn’t I fail the linking then?
> Definition generators are the solution to this. You can get the address of symbols defined by Module A while A still contains undefined external references, however definitions for those undefined external references must be supplied by a definition generator or Module A will immediately fail linking.
I would like to use the definition generator, because this is what I tried in the past (and aligns a bit more with our old loader). This is the call to ExecutionSession.lookup right? However this only worked when I was able to provide those addresses. I have no problem to fail the linking if I have a chance to do it again in a second or third or… call. However, when I tried this it seemed like I only got one chance. The “tyrToGenerate” function was only called in my first call to lookup, which failed linking cause I couldn’t provide an address.
So is there any chance to get back at linking or kinda…. Restarting it?
Our architecture allows that a module which is referenced by another module might appear a bit lit, which means that the code has to wait until the other one is present.
Still so many questions… Sorry! The thing is, everything works so fine when all those symbols are present, but when they aren’t I run into various troubles. However our architecture basically allows to throw modules at the system and keep nagging it until they are all resolved, while having the addresses for all the symbols inside the module already present, which is why the IR approach is kiiinda difficult for me :/
Thank you again and again Lang!