Julia Memory Management

[Chris Rackauckas]

I see mentions like this one every once in awhile:

"D language is a special case, as it has GC, but it's also optional (as with Julia)"

Is GC optional? I thought the only way to discard something from memory was to set it to zero and call garbage control (which then runs the whole garbage control). Is there a more targeted way to delete things? If it's not already available, it seems like it would be useful for code focusing performance as an option.

[Yichao Yu]

On Thu, May 26, 2016 at 11:06 AM, Chris Rackauckas <rack...@gmail.com> wrote:
> I see mentions like this one every once in awhile:
>
> "D language is a special case, as it has GC, but it's also optional (as with
> Julia)"
>
> Is GC optional?

No, Not for julia objects.

> I thought the only way to discard something from memory was
> to set it to zero and call garbage control (which then runs the whole
> garbage control). Is there a more targeted way to delete things?

No, Not for julia objects.

> If it's not
> already available, it seems like it would be useful for code focusing
> performance as an option.

This will actually **not** improve performance most of the time and
can actually hurt performance a lot.
The only case I can think of that can have better performance is
manually managing an object pool.
This might improve latency or memory usage. All of the advantage can
be obtained by improving the GC itself, which is the preferred way.
This feature will also be extremely unsafe and can break many
assumptions made in the runtime.

[Ford Ox]

I wonder there will be ever be so fast Garbage Collector, that reference counting, manual allocation / freeing and all that c++ stuff will be obsolete...

[Chris Rackauckas]

Thanks, that's good to know. Can you explain a little bit about why it would hurt performance? 

[Yichao Yu]

This is the wrong question to ask. As I mentioned already, a tracing
GC is usually much faster (in throughput) than any other GC techniques
that I know of (except not freeing memory). Even the simple GC julia
has outperform the best malloc implementation I can find by a large
margin.

The right GC techniques to use is usually a trade off between
throughput and latency.

[Yichao Yu]

On Thu, May 26, 2016 at 11:44 AM, Chris Rackauckas <rack...@gmail.com> wrote:
> Thanks, that's good to know. Can you explain a little bit about why it would
> hurt performance?

See my response on an old thread.
https://groups.google.com/forum/?fromgroups=#!searchin/julia-users/Suspending$20Garbage$20Collection$20for$20Performance...good$20idea$20or$20bad$20idea$3F/julia-users/6_XvoLBzN60/nkB30SwmdHQJ

In short, the key points are that

1. Freeing memory is not free.

You need to do work so that you can reclaim it later, (why would
you free it otherwise....)

2. Doing the work incrementally is typically more expensive than doing
the work in batches.

This is where the trade off between latency and throughput comes in.

These being said, there are of course cases that you can construct
with manually tweaked memory management that beats the best GC which
has to guess the best strategy (and finding such example should be
easier for a less optimized GC). However, I think currently the
unsafeness, the likelihood of misuse and the impact on other part of
the runtime doesn't really justify introducing manual memory
management of julia objects.

[Chris Rackauckas]

Ahh, that makes sense. Thanks for clearing that up. 

[Jeff Bezanson]

I love this topic; can't resist chiming in. There is indeed a lot of
misinformation out there about "manual memory management". I believe
all the gains (if any) to be had from manual memory management come
from (1) requesting stack allocation (it can be hard for a compiler to
prove that the lifetime of an object is limited enough for this to
work), (2) reusing space efficiently by overwriting it.

(1) mostly applies to small objects. We handle it by providing
immutable objects, which are much easier to automatically stack
allocate. We can't yet stack allocate all immutable objects, but work
has been done on it and we'll get there eventually. Also due to
stack-bounded lifetimes, these objects are likely to become young
garbage that GC is especially good at handling.

(2) is possible in any language with mutation. Overwriting data is a
form of manual memory management (you're saying you don't need the old
version of the data any more). In Julia folks get speedups from this
all the time. While I'd love it if we could get that level of
performance automatically, in the meantime this is a highly effective
form of manual memory management that's fully compatible with GC. So I
get annoyed when somebody says languages should have manual memory
management *instead of* GC (which fortunately nobody argued here). If
you're willing to use manual memory management techniques, a GC isn't
going to slow you down! And Yichao is right that for a given heap
allocation workload, GC handily beats malloc/free (since the gains
come from reusing space and avoiding heap allocation, not from
malloc/free being fast). I suspect a lot of the confusion is due to
languages with GC tending not to offer in-place algorithms and other
kinds of support for mutation, even though they could.

[Ford Ox]

This is the wrong question to ask

 

I wanted to write 'good enough' but then I discarded it since its too broad term. It was a mistake.

I was wondering, whether there will ever be so well rounded GC, that ... "reference counting, manual allocation / freeing and all that c++ stuff will be obsolete."

Reacting to Jeff post

I always hear from people working in game industry (graphics) that GC languages can't be used, since it always causes some minor lags. Is that another misinformation?

[Stefan Karpinski]

This is pointlessly philosophical and off topic at this point.

[Oscar Blumberg]

When video game people say that they usually mean "tracing gc" instead of "gc" and it's about the latency, not the throughput, which Yichao explained.

A game has a soft deadline every few tens of milliseconds to render to the screen and gc pauses can make the experience miserable.

Interestingly a lot of games actually have a tracing gc that handles some objects, either because they embed a managed scripting language for gameplay or even because they have a C++ gc (I'm looking at you Unreal Engine).

The key is to keep the number of managed object small-ish and collect them incrementally every frame.
The general saying "gc is a no go for real-time games" usually comes with a strawman where every single 3 component vector is heap allocated.

[Páll Haraldsson]

On Thursday, May 26, 2016 at 3:06:04 PM UTC, Chris Rackauckas wrote:

I see mentions like this one every once in awhile:

"D language is a special case, as it has GC, but it's also optional (as with Julia)"

I've been saying something like this [and actually googled, it wasn't me..]

I may have been spreading misinformation, only recently understanding D better in what (better) sense GC is "optional" there:


D has GC by default, but better handling (@nogc) for being sure only manual memory allocation is allowed in a function (and I guess that function is disallowed to call a funtion not marked as such).

In D, if not all your functions are marked as such, then it's about as bad to disable the GC [for a long time] as doing so in Julia is.

There are two recent thread here on real-time and avoiding allocations.

You can use Libc.malloc and Libc.free directly in Julia (or call C and use manual memory management that way indirectly). That will only get you bytes, not objects.

You can use Ptr on those bytes and "reinterpret" (does it only allow some types, not "objects" in general?).. It's probably not easy(?) to work that way outside of the GC; I'm not sure what it involves using those bytes as objects to get it to be easier, except by letting the GC take over, just as it can with C allocated memory.


Maybe I should try harder to convince other people (and myself) that Julia works very well (even for non-hard real-time) just as it is with the GC enabled, including for games. I just like pointing out you have the same powers as C/C++ without (or with) calling C/C++.

Reference counting is predictably slow, and you can get an avalanche of deallocations. Not doing the deallocations/GC activity can be faster when it's not happening, but in the end you might have an even bigger avalanche with naive variants of GC (I guess in 0.3), but not an avalanche in incremental as in 0.4.


Interesting quote from Oscar: "Interestingly a lot of games actually have a tracing gc that handles some objects,[..] or even because they have a C++ gc (I'm looking at you Unreal Engine)."

I know about GC e.g. with Lua, with games engines, but not within game engines/C++ actually being used, even though I know of the possibility: Boehm-style GC optional in C/C++ (even allowed by the C++ standard):

https://github.com/ivmai/bdwgc


[Love this quote from Jeff, just hadn't thought of it that way: "Overwriting data is a form of manual memory management (you're saying you don't need the old
version of the data any more). In Julia folks get speedups from this all the time."]