[llvm-dev] Pardon the newbie question
pawel k. via llvm-dev
Min-Yih Hsu via llvm-dev
1. You can use `LLVM_PARALLEL_LINK_JOBS=N` (also a cmake variable) to limit the amount of parallel linker jobs to save some memory.
2. Build libraries as shared library via `BUILD_SHARED_LIBS=ON` CMake variable can dramatically speed up the linking time and save you some memory.
3. Since you’re building a Debug build (and you’re building on Linux), `LLVM_USE_SPLIT_DWARF` can dramatically reduce the size of debug info, which, to some extend, also save you some memory during link time.
Best,
-Min
> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
via llvm-dev
configuration, and add `-DCMAKE_C_FLAGS=-g1 -DCMAKE_CXX_FLAGS=-g1`
The `-g1` means you get debug info for line numbers, although not really
types and variables; but this is enough for symbolized backtraces after
a crash or assertion. This is what I use normally.
--paulr
> -----Original Message-----
> From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of Min-Yih Hsu
> via llvm-dev
> Sent: Thursday, April 15, 2021 5:37 PM
> To: pawel...@gmail.com
> Cc: llvm...@lists.llvm.org
> Subject: Re: [llvm-dev] Pardon the newbie question
>
> You can use `LLVM_USE_LINKER=lld` CMake variable to adopt LLD (to build
> LLVM). And yes, LLD takes less memory and runs faster. Here are some other
> tips to save memory:
> 1. You can use `LLVM_PARALLEL_LINK_JOBS=N` (also a cmake variable) to
> limit the amount of parallel linker jobs to save some memory.
>
> 2. Build libraries as shared library via `BUILD_SHARED_LIBS=ON` CMake
> variable can dramatically speed up the linking time and save you some
> memory.
>
> 3. Since you’re building a Debug build (and you’re building on Linux),
> `LLVM_USE_SPLIT_DWARF` can dramatically reduce the size of debug info,
> which, to some extend, also save you some memory during link time.
>
> Best,
> -Min
> >
> > Hello,
> > Im trying to build trunk clang in debug version on oldish ubuntu with
> low mem. Linking lli takes ages and fails on low mem. Is there a chance
> building would succeed if i used lld instead of ld? If so is there an
> option either to force lld or whole clang toolchain use in cmake instead
> of default gcc (both gcc and clang are avail on system)? Otherwise I think
> ill stick with release.
> >
> > Best regards,
> > Pawel Kunio
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm...@lists.llvm.org
> bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!ta_FURsYjV22Mf_11nioL-
> 1WoGxYffvrC81QhwlPVI--_YkhJyoS4dEzUAzddJdt3w$
> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!ta_FURsYjV22Mf_11nioL-
> 1WoGxYffvrC81QhwlPVI--_YkhJyoS4dEzUAzddJdt3w$
pawel k. via llvm-dev
pawel k. via llvm-dev
Joerg Sonnenberger via llvm-dev
Your best options for that are using DWARF fusion
(-DLLVM_USE_SPLIT_DWARF) or as separate shared libraries
(-DBUILD_SHARED_LIBS). The former will punish the gdb load time and the
latter will penalize execution speed.
Joerg
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
David Blaikie via llvm-dev
<llvm...@lists.llvm.org> wrote:
>
> Hiya,
>
> Enormous thanks for all suggests and extensive too. Testing splitdwarf asap.
>
> Makes me think what i thought previously. We need dwarf go pdbish way.
Split DWARF provides something like this, though with some different
tradeoffs - for instance Split DWARF is easier to use in a distributed
build (since the compiler doesn't have to communicate with a separate
server while it's compiling).
> Separate dblike possibly adressable by url.
Symbol servers are another problem - even using classic DWARF you can
then strip the main binary, keep the unstripped binary and put that
unstripped binary on some kind of symbol server system (I'm not sure
if gdb supports something like that natively, but could be done
without DWARF changes).
> I was hacking pdb techno before it got sexy. I know a bit how it is organized and could try to help with architecting or designing such solution. Loved dbinfo on vstudio. Gdbish not so much.
What sort of problems have you had with gdb or the DWARF debugging
workflow more generally?
> And i was slaving on my corpo cotton plantation passively using gdb for about a decade.
(analogies to slavery aren't suitable for this community)
- Dave
pawel k. via llvm-dev
pawel k. via llvm-dev
David Blaikie via llvm-dev
>
> Ok for gdb, load time was major pain.
Ah, for large binaries you probably want to compile with
-ggnu-pubnames -Wl,--gdb-index (you must compile with this for Split
DWARF - it relies on the index) that takes, for instance, gdb startup
time debugging clang from 3 minutes to about 3 seconds. (lldb does a
better job lazy loading DWARF - so faster startup time even without an
index (& it can't use the gdb index anyway) trading off some
operations later will be slower as things are lazy loaded later on)
> Type casting was patchy some types missing etc,
Hard to say what's at work there - happy to speculate/help if you have
a specific/isolated example.
> some var tracking especially under optimizations was tricky for example having to cast registers because info this var is stored in this register was missing
Yeah, that's less about the workflow/technology, but more as we like
to say, the "quality of implementation" - keeping track of variables
under optimization is tricky, and different compilers do it
well/poorly - DWARF certainly offers a good variety of ways to express
these things (and gaining more as DWARF improves - it's certainly not
"complete", but I don't think it ever can be - there'll always be more
cunning optimizations and cunning ways to try to recover the value of
user variables under those optimizations).
> hmm also one mire thing that could minimize debug info size could be shared types section with removing local copies per compilation unit.
Yep - that can already be done with DWARF type units
(-fdebug-types-section), but also various strategies to reduce
emitting the types in the first place (hopefully we'll be enabling the
constructor homing strategy by default soon which should reduce
duplicate type information significantly)
> Also global vars etc could be stored in shared segment. Also we could think of somehow indexing the db.
Yeah, as mentioned above, gdb (& lld and gold) support the gdb_index
format and that can be generated at link time with -Wl,--gdb-index
(objects need to be compiled with -ggnu-pubnames (that comes by
default if you specify -gsplit-dwarf I think... ) for this to work).
And there is a DWARFv5 debug_names section, but it's not fully
supported yet (we don't have any linker that supports merging
.debug_names from object files into a unified .debug_names in the
linked binary - nor any debuggers that can consume the .debug_names
index).
pawel k. via llvm-dev
David Blaikie via llvm-dev
>
> Is constructor homing this featurette where class di is emitted only for compunits with reference to constructor?
Yep - both Clang and GCC already do something equivalent for any type
with virtual functions - the debug info for that type is emitted only
where the vtable is emitted (so called "vtable homing") so it's not
without precedent. Yeah, it means potentially missing any type that
isn't constructed (types used only to hold a bunch of static members,
for instance) - but perhaps such a type isn't needed anyway. This
isn't done for types with trivial constructors - since they don't
generate code, etc.
> Id speculate therell be problems with it as i understand the idea by now.
Certainly open to any critiques/bugs/etc. It's currently accessible
under -fuse-ctor-homing
https://www.mail-archive.com/cfe-c...@lists.llvm.org/msg218269.html
> Shared type table feels bit more complex on impl side fir merging etc but feels stronger.
Tradeoffs to be sure - anything that's shared during compile time's
not really usable in the build environment (massively distributed) I
support, but can work for others (like MSVC does). C++ modules (both
Clang Header Modules and C++20 standard modules) offer another way to
"home" debug info - since modules are then a first class build product
they can be used to home debug info quite reliably (template
instantiations may be duplicated - since they may be instantiated in
multiple unrelated places, but even then any module depending on a
module with an insntantiation need not re-do that instantiation).
- Dave
pawel k. via llvm-dev
pawel k. via llvm-dev
kuter dinel via llvm-dev
pawel k. via llvm-dev
David Blaikie via llvm-dev
>
> Hello,
> Splitdwarf worked nicely for me. Big thaks to all contributing and biy there were many useful hints.
>
> I think that would be useful. I can try to collect them all and brief each and then somebody could apply the styling if need be or if just pure text paragraph would suffice I can prepare some file like that. I especially liked lineinfo+funcaddrs only option. Very useful for stacktraces and asserts if not planning to use gdb.
I think most of this is written up here:
https://llvm.org/docs/GettingStarted.html#common-problems
Though looks like it could have a bullet point for -g1-for-backtraces too.