Turn off Garbage collection?
152 views
Skip to first unread message
Tobias Pfeiffer
May 30, 2016, 3:34:19 AM5/30/16
to elixir-l...@googlegroups.com
Hi everyone,
is there a way in Elixir/Erlang to turn off the Garbage collection? I've
searched and what I found so far is :erlang.garbage_collect to force
garbage collection. http://erlang.org/doc/man/erlang.html#garbage_collect-0
Is there any way to turn it off completely?
Why would I want to do that? I'm working on a benchmarking tool and I
don't want garbage collection to mess with my measured execution times.
Any hints welcome :)
Tobi
--
http://www.pragtob.info/
is there a way in Elixir/Erlang to turn off the Garbage collection? I've
searched and what I found so far is :erlang.garbage_collect to force
garbage collection. http://erlang.org/doc/man/erlang.html#garbage_collect-0
Is there any way to turn it off completely?
Why would I want to do that? I'm working on a benchmarking tool and I
don't want garbage collection to mess with my measured execution times.
Any hints welcome :)
Tobi
--
http://www.pragtob.info/
Robert Virding
May 30, 2016, 8:36:58 AM5/30/16
to elixir-lang-talk
Sorry to be slightly sarcastic here but if you were to be able to turn GC then:
- How relevant would your benchmarks then be to "real" systems?
- How would benchmark the system crashing as it ran out of memory? (couldn't help myself here)
Robert
- How relevant would your benchmarks then be to "real" systems?
- How would benchmark the system crashing as it ran out of memory? (couldn't help myself here)
Robert
Andrei Dziahel
May 30, 2016, 9:38:37 AM5/30/16
to elixir-l...@googlegroups.com
Hi,
Did you figure out how exactly does GC mess with your execution times and if so, could you share your outcomes here?
--
http://www.pragtob.info/
--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/574BECF6.1060309%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
Regards,
Andrei Dziahel
Greg Young
May 30, 2016, 9:44:59 AM5/30/16
to elixir-l...@googlegroups.com
Wouldn't this be a very bad idea for any kind of benchmarking?
I imagine a runner in a race. He has special shoes that make him go a
bit faster but for every step he takes a piece of the shoe falls off,
occasionally (every 5km) he has to change his shoes. So for our test
of his speed we measure a 3km measurement. Unfortunately in the actual
race he has to run 25km.
Cheers,
Greg
I imagine a runner in a race. He has special shoes that make him go a
bit faster but for every step he takes a piece of the shoe falls off,
occasionally (every 5km) he has to change his shoes. So for our test
of his speed we measure a 3km measurement. Unfortunately in the actual
race he has to run 25km.
Cheers,
Greg
> --
> You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/574BECF6.1060309%40gmail.com.
> For more options, visit https://groups.google.com/d/optout.
--
Studying for the Turing test
> You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/574BECF6.1060309%40gmail.com.
> For more options, visit https://groups.google.com/d/optout.
--
Onorio Catenacci
May 30, 2016, 10:25:08 AM5/30/16
to elixir-lang-talk
How much information will your benchmarks give you if you remove a major performance-affecting component which will be present in production? Seems like it would be significantly less useful.
Scott Ribe
May 30, 2016, 2:24:13 PM5/30/16
to elixir-l...@googlegroups.com
On May 30, 2016, at 8:25 AM, Onorio Catenacci <cate...@gmail.com> wrote:
>
> How much information will your benchmarks give you if you remove a major performance-affecting component which will be present in production?
It could give you a basis for quantifying the performance effect of that component, thus guiding the answer to whether tuning efforts should focus on that component or elsewhere.
>
> How much information will your benchmarks give you if you remove a major performance-affecting component which will be present in production?
--
Scott Ribe
scott...@elevated-dev.com
http://www.elevated-dev.com/
https://www.linkedin.com/in/scottribe/
(303) 722-0567 voice
Gleb Arshinov
May 30, 2016, 5:48:53 PM5/30/16
to elixir-l...@googlegroups.com
That's a good analogy. And it's describes the reason why being able
to turn off GC is useful for performance optimization.
It's more obvious when you are profiling. If you don't turn off GC
when profiling, the full cost of GC will be attributed to the last
function that triggered GC, not functions that allocated memory
before. The straw that broke camel's back, e.g. "left leg step at
kilometer 5" in your analogy. While in reality it needs to allocated
over left and right steps over the previous 5 kilometers.
When doing performance work in a GC language the cost of a piece of
code is a pair of:
* CPU/clock time it take to run (in seconds)
* and GC pressure it generates (in # of allocations, total size of
allocations, etc.)
You can convert GC pressure to seconds. But the conversion rate may
differ greatly between a micro-benchmark and production environment.
E.g. 10-100ms in Ruby <2.1. So you need to keep GC pressure
separate, and have a good understanding of how GC works to compare
these pairs.
High level, comparing a benchmark w/ GC on and off is a good way to
see if GC time is a significant portion of runtime.
Having said this I have no idea how BEAM GC works and how significant
it is for performance.
Best regards,
Gleb
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAC9RQthoVqFCjYunRX7eMr9nF7tybB5tyMKP43GkOT9WoaPSBQ%40mail.gmail.com.
to turn off GC is useful for performance optimization.
It's more obvious when you are profiling. If you don't turn off GC
when profiling, the full cost of GC will be attributed to the last
function that triggered GC, not functions that allocated memory
before. The straw that broke camel's back, e.g. "left leg step at
kilometer 5" in your analogy. While in reality it needs to allocated
over left and right steps over the previous 5 kilometers.
When doing performance work in a GC language the cost of a piece of
code is a pair of:
* CPU/clock time it take to run (in seconds)
* and GC pressure it generates (in # of allocations, total size of
allocations, etc.)
You can convert GC pressure to seconds. But the conversion rate may
differ greatly between a micro-benchmark and production environment.
E.g. 10-100ms in Ruby <2.1. So you need to keep GC pressure
separate, and have a good understanding of how GC works to compare
these pairs.
High level, comparing a benchmark w/ GC on and off is a good way to
see if GC time is a significant portion of runtime.
Having said this I have no idea how BEAM GC works and how significant
it is for performance.
Best regards,
Gleb
Onorio Catenacci
May 30, 2016, 6:11:00 PM5/30/16
to elixir-l...@googlegroups.com
That's a good point Scott. It still strikes me as probably pointless since as far as I know the GC is not something which can be overridden. So knowing the performance bottleneck is in GC doesn't seem to help much.
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-talk/lX0ve2YNi64/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CACZNi59evCGuyTcs25QLb%3DScA%3DpabrX%2Bh__NqMf%3D%3DFHJNt_ydA%40mail.gmail.com.
Greg Young
May 30, 2016, 6:18:01 PM5/30/16
to elixir-l...@googlegroups.com
They way people normally do this is to:
a) measure at 3km instead of 25km or attempt to amortize the costs
over longer runs say 500km while recognizing them (histograms help
here)
b) measure the time of changing shoes and discount it from the measurements
The nice thing about both of these answers is that it forces you to
recognize the problem with the shoes.
Cheers,
Greg
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CACZNi59evCGuyTcs25QLb%3DScA%3DpabrX%2Bh__NqMf%3D%3DFHJNt_ydA%40mail.gmail.com.
a) measure at 3km instead of 25km or attempt to amortize the costs
over longer runs say 500km while recognizing them (histograms help
here)
b) measure the time of changing shoes and discount it from the measurements
The nice thing about both of these answers is that it forces you to
recognize the problem with the shoes.
Cheers,
Greg
Dmitry Belyaev
May 30, 2016, 7:29:18 PM5/30/16
to elixir-l...@googlegroups.com, Tobias Pfeiffer
Returning to the available choices rather than discussion of necessity.
You can try to use erlang:spawn_opt http://erlang.org/doc/man/erlang.html#spawn_opt-2 setting fullsweep_after and min_heap_size to high values to reduce chances of garbage collection.
You should also ask this in erlang-questions list as there are some people who know internals of beam vm but don't track questions in this mailing list.
You can try to use erlang:spawn_opt http://erlang.org/doc/man/erlang.html#spawn_opt-2 setting fullsweep_after and min_heap_size to high values to reduce chances of garbage collection.
You should also ask this in erlang-questions list as there are some people who know internals of beam vm but don't track questions in this mailing list.
Best wishes,
Dmitry Belyaev
Dmitry Belyaev
Tobias Pfeiffer
May 31, 2016, 5:57:53 AM5/31/16
to elixir-l...@googlegroups.com
Hi there all,
somehow I didn't see replies in my mail client before.
The reason I want to (optionally) turn off GC is as especially for micro
benchmarking the GC will only occur for one of the runs and that run
will then have a significant increase in run time. This causes outliers
and makes the average less meaningful and the standard deviation way
higher than it ought to be.
I agree that for more "macro" style benchmarks I probably wouldn't want
to turn off GC because it's an important point there.
as one example when running a very fast benchmark the results are
usually 80-100 nanoseconds, but some end up being 5000+ messing with the
end result.
I'll also try to fend those off using statistical methods (like median)
but it's not exactly my strong suit (yet) :)
also interesting to see the points here against turning off GC. In Ruby
it's rather standard practice, benchmark-ips does it and the Ruby
performance optimization book explicitly promotes it as a best practice.
Thanks for the input everyone!
Tobi
http://www.pragtob.info/
somehow I didn't see replies in my mail client before.
The reason I want to (optionally) turn off GC is as especially for micro
benchmarking the GC will only occur for one of the runs and that run
will then have a significant increase in run time. This causes outliers
and makes the average less meaningful and the standard deviation way
higher than it ought to be.
I agree that for more "macro" style benchmarks I probably wouldn't want
to turn off GC because it's an important point there.
as one example when running a very fast benchmark the results are
usually 80-100 nanoseconds, but some end up being 5000+ messing with the
end result.
I'll also try to fend those off using statistical methods (like median)
but it's not exactly my strong suit (yet) :)
also interesting to see the points here against turning off GC. In Ruby
it's rather standard practice, benchmark-ips does it and the Ruby
performance optimization book explicitly promotes it as a best practice.
Thanks for the input everyone!
Tobi
Tobias Pfeiffer
May 31, 2016, 5:59:05 AM5/31/16
to Dmitry Belyaev, elixir-l...@googlegroups.com
Thank you very much for that pointer, that looks helpful :) I'll also
make sure to ask over at erlang-questions!
Tobi
--
http://www.pragtob.info/
make sure to ask over at erlang-questions!
Tobi
http://www.pragtob.info/
Robert Virding
Jun 2, 2016, 11:01:34 AM6/2/16
to elixir-lang-talk, be.d...@gmail.com
A thing to realise is that GC times are generally very short as you are only collecting one process at a time. Sorry I missed whether you are measuring cpu execution time or clocktime. Another way is to use the tracing facility to keep track of what is happening in your process, when it is being scheduled in and out and when it is garbage collecting. This will give you much better information about what is going on in your process.
Robert
Robert
Tobias Pfeiffer
Jun 2, 2016, 4:20:11 PM6/2/16
to elixir-l...@googlegroups.com
Didn't know about the run times, good to know they are short but I guess
I'll learn more about that as I go. However small they are, when I want
to benchmark different map implementations or something like that with
maybe smaller lists than one GC is an enormous hit. :)
> <https://groups.google.com/d/msgid/elixir-lang-talk/4f1ed757-e5c3-4591-986d-5d70d8a022e8%40googlegroups.com?utm_medium=email&utm_source=footer>.
I'll learn more about that as I go. However small they are, when I want
to benchmark different map implementations or something like that with
maybe smaller lists than one GC is an enormous hit. :)
> <javascript:>> wrote:
> >
> > Hi everyone,
> >
> > is there a way in Elixir/Erlang to turn off the Garbage
> collection? I've
> > searched and what I found so far is :erlang.garbage_collect to
> force
> > garbage collection.
> http://erlang.org/doc/man/erlang.html#garbage_collect-0
> <http://erlang.org/doc/man/erlang.html#garbage_collect-0>
> >
> > Is there any way to turn it off completely?
> >
> > Why would I want to do that? I'm working on a benchmarking
> tool and I
> > don't want garbage collection to mess with my measured
> execution times.
> >
> > Any hints welcome :)
> > Tobi
> >
> >
> > --
> > Best wishes,
> > Dmitry Belyaev
>
> --
> http://www.pragtob.info/
>
> >
> > Hi everyone,
> >
> > is there a way in Elixir/Erlang to turn off the Garbage
> collection? I've
> > searched and what I found so far is :erlang.garbage_collect to
> force
> > garbage collection.
> http://erlang.org/doc/man/erlang.html#garbage_collect-0
> <http://erlang.org/doc/man/erlang.html#garbage_collect-0>
> >
> > Is there any way to turn it off completely?
> >
> > Why would I want to do that? I'm working on a benchmarking
> tool and I
> > don't want garbage collection to mess with my measured
> execution times.
> >
> > Any hints welcome :)
> > Tobi
> >
> >
> > --
> > Best wishes,
> > Dmitry Belyaev
>
> --
> http://www.pragtob.info/
>
> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elixir-lang-ta...@googlegroups.com
> <mailto:elixir-lang-ta...@googlegroups.com>.
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elixir-lang-ta...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-talk/4f1ed757-e5c3-4591-986d-5d70d8a022e8%40googlegroups.com
> <https://groups.google.com/d/msgid/elixir-lang-talk/4f1ed757-e5c3-4591-986d-5d70d8a022e8%40googlegroups.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages