[Haskell-cafe] ghc-prof-options and libraries on Hackage
20 views
Skip to first unread message
Oliver Charles
Sep 21, 2014, 2:13:39 PM9/21/14
to Haskell Cafe
Hi all,
I'd like to start a conversation around the ghc-prof-options option that can be used in Cabal-ized libraries. As you may know, this option specifies extra GHC options that are used when --enable-library-profiling is enabled. This is very convenient for local development, but I argue that it can be counter-productive for releases onto Hackage.
To provide an example, I'm currently working on a little game engine that uses JuicyPixels to load images. I have a problem in my code that needs optimizing, but the current state of things results in profiles that are very difficult to work with. JuicyPixels specifies -auto-all in its cabal file, which means I have no alternative but to profile JuicyPixels code. In this scenario, the bottleneck is actually within my FRP game loop and nothing to do with image loading! As a result, the profiles are fairly useless to me.
Roman Cheplyaka also points out that by doing this, profiles are skewed - -auto-all is not free, so we actually pay in runtime performance every time a library does this.
I would like to hear from others if we should consider managing this option a little more. My personal feeling is that this flag shouldn't be used unless in local development, so guarded by an off-by-default build flag. I think cabal should also warn authors who are using this flag, and encourage them to place a guard around this option.
Thoughts?
- ocharlesI'd like to start a conversation around the ghc-prof-options option that can be used in Cabal-ized libraries. As you may know, this option specifies extra GHC options that are used when --enable-library-profiling is enabled. This is very convenient for local development, but I argue that it can be counter-productive for releases onto Hackage.
To provide an example, I'm currently working on a little game engine that uses JuicyPixels to load images. I have a problem in my code that needs optimizing, but the current state of things results in profiles that are very difficult to work with. JuicyPixels specifies -auto-all in its cabal file, which means I have no alternative but to profile JuicyPixels code. In this scenario, the bottleneck is actually within my FRP game loop and nothing to do with image loading! As a result, the profiles are fairly useless to me.
Roman Cheplyaka also points out that by doing this, profiles are skewed - -auto-all is not free, so we actually pay in runtime performance every time a library does this.
I would like to hear from others if we should consider managing this option a little more. My personal feeling is that this flag shouldn't be used unless in local development, so guarded by an off-by-default build flag. I think cabal should also warn authors who are using this flag, and encourage them to place a guard around this option.
Thoughts?
Johan Tibell
Sep 21, 2014, 4:46:01 PM9/21/14
to Oliver Charles, haskell-cafe
Agreed. Cabal should warn.
_______________________________________________
Haskell-Cafe mailing list
Haskel...@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Eric Seidel
Sep 21, 2014, 4:52:25 PM9/21/14
to Haskell Cafe
> To provide an example, I'm currently working on a little game engine that
> uses JuicyPixels to load images. I have a problem in my code that needs
> optimizing, but the current state of things results in profiles that are
> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
> file, which means I have no alternative but to profile JuicyPixels code. In
> this scenario, the bottleneck is actually within my FRP game loop and
> nothing to do with image loading! As a result, the profiles are fairly
> useless to me.
While in this case the extra profiling data may be useless, it seems to me that in general you won't know a priori. I would prefer to have as much data available as possible, and then filter it.
> uses JuicyPixels to load images. I have a problem in my code that needs
> optimizing, but the current state of things results in profiles that are
> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
> file, which means I have no alternative but to profile JuicyPixels code. In
> this scenario, the bottleneck is actually within my FRP game loop and
> nothing to do with image loading! As a result, the profiles are fairly
> useless to me.
HPC works sort of like this, it gathers coverage data for all of the modules compiled with -fhpc, but has command-line options to only report coverage for certain modules. Perhaps GHC's rts could add a similar flag to only report profiling data from certain modules or functions?
Ivan Lazar Miljenovic
Sep 21, 2014, 6:58:41 PM9/21/14
to Eric Seidel, Haskell Cafe
On 22 September 2014 06:52, Eric Seidel <ese...@cs.ucsd.edu> wrote:
>> To provide an example, I'm currently working on a little game engine that
>> uses JuicyPixels to load images. I have a problem in my code that needs
>> optimizing, but the current state of things results in profiles that are
>> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
>> file, which means I have no alternative but to profile JuicyPixels code. In
>> this scenario, the bottleneck is actually within my FRP game loop and
>> nothing to do with image loading! As a result, the profiles are fairly
>> useless to me.
>
> While in this case the extra profiling data may be useless, it seems to me that in general you won't know a priori. I would prefer to have as much data available as possible, and then filter it.
Agreed: I've often had to do a "cabal unpack <foo>", edit the .cabal
>> To provide an example, I'm currently working on a little game engine that
>> uses JuicyPixels to load images. I have a problem in my code that needs
>> optimizing, but the current state of things results in profiles that are
>> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
>> file, which means I have no alternative but to profile JuicyPixels code. In
>> this scenario, the bottleneck is actually within my FRP game loop and
>> nothing to do with image loading! As a result, the profiles are fairly
>> useless to me.
>
> While in this case the extra profiling data may be useless, it seems to me that in general you won't know a priori. I would prefer to have as much data available as possible, and then filter it.
file to add the profiling options and re-install that package (which
is admittedly easier/nicer now with sandboxes).
--
Ivan Lazar Miljenovic
Ivan.Mi...@gmail.com
http://IvanMiljenovic.wordpress.com
John Lato
Sep 21, 2014, 7:28:57 PM9/21/14
to Eric Seidel, Haskell Cafe
On Mon, Sep 22, 2014 at 4:52 AM, Eric Seidel <ese...@cs.ucsd.edu> wrote:
> To provide an example, I'm currently working on a little game engine that
> uses JuicyPixels to load images. I have a problem in my code that needs
> optimizing, but the current state of things results in profiles that are
> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
> file, which means I have no alternative but to profile JuicyPixels code. In
> this scenario, the bottleneck is actually within my FRP game loop and
> nothing to do with image loading! As a result, the profiles are fairly
> useless to me.
While in this case the extra profiling data may be useless, it seems to me that in general you won't know a priori. I would prefer to have as much data available as possible, and then filter it.
HPC works sort of like this, it gathers coverage data for all of the modules compiled with -fhpc, but has command-line options to only report coverage for certain modules. Perhaps GHC's rts could add a similar flag to only report profiling data from certain modules or functions?
I am sympathetic to this concern, but the main problem here is that -fprof-auto isn't free. It can interfere quite heavily with optimization passes, meaning that it ends up attributing much higher costs to certain functions than they would have in a normal compilation path, actively misleading users. I for one would have no confidence that profiles were pointing to the correct place if everything were built with -fprof-auto from the start. It's much better to start with just a few high-level cost-centers and drill down from there. This can be a tedious process, but it's much more reliable. As an alternative, compiling libraries with -fprof-auto-exported is fairly reasonable IMHO.
John L.
Andrew Cowie
Sep 21, 2014, 9:50:29 PM9/21/14
to haskel...@haskell.org
On Mon, 2014-09-22 at 07:28 +0800, John Lato wrote:
> compiling libraries with -fprof-auto-exported
That's what I was going to pipe up with. In my usage, having
dependencies at -fprof-auto-exported seems to work quite well. On the
rare occasion that I need *more*, then off to `cabal unpack` we go, but
that's a better trade off I think.
AfC
Sydney
> compiling libraries with -fprof-auto-exported
That's what I was going to pipe up with. In my usage, having
dependencies at -fprof-auto-exported seems to work quite well. On the
rare occasion that I need *more*, then off to `cabal unpack` we go, but
that's a better trade off I think.
AfC
Sydney
Daniel Trstenjak
Sep 22, 2014, 2:44:37 AM9/22/14
to haskel...@haskell.org
On Mon, Sep 22, 2014 at 07:28:41AM +0800, John Lato wrote:
> As an alternative, compiling libraries with -fprof-auto-exported is fairly reasonable IMHO.
you could just build everything in a cabal sandbox with the desired options.
Greetings,
Daniel
Michael Snoyman
Sep 22, 2014, 2:59:24 AM9/22/14
to Johan Tibell, haskell-cafe
On Sun, Sep 21, 2014 at 11:45 PM, Johan Tibell <johan....@gmail.com> wrote:
+1.
Agreed. Cabal should warn.
+1.
Roman Cheplyaka
Sep 22, 2014, 3:25:36 AM9/22/14
to Ivan Lazar Miljenovic, Eric Seidel, Haskell Cafe
On 22/09/14 01:58, Ivan Lazar Miljenovic wrote:
> On 22 September 2014 06:52, Eric Seidel <ese...@cs.ucsd.edu> wrote:
>>> To provide an example, I'm currently working on a little game engine that
>>> uses JuicyPixels to load images. I have a problem in my code that needs
>>> optimizing, but the current state of things results in profiles that are
>>> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
>>> file, which means I have no alternative but to profile JuicyPixels code. In
>>> this scenario, the bottleneck is actually within my FRP game loop and
>>> nothing to do with image loading! As a result, the profiles are fairly
>>> useless to me.
>>
>> While in this case the extra profiling data may be useless, it seems to me that in general you won't know a priori. I would prefer to have as much data available as possible, and then filter it.
>
> Agreed: I've often had to do a "cabal unpack <foo>", edit the .cabal
> file to add the profiling options and re-install that package (which
> is admittedly easier/nicer now with sandboxes).
Why would you need to edit the cabal file? Are you aware of --ghc-options?
> On 22 September 2014 06:52, Eric Seidel <ese...@cs.ucsd.edu> wrote:
>>> To provide an example, I'm currently working on a little game engine that
>>> uses JuicyPixels to load images. I have a problem in my code that needs
>>> optimizing, but the current state of things results in profiles that are
>>> very difficult to work with. JuicyPixels specifies -auto-all in its cabal
>>> file, which means I have no alternative but to profile JuicyPixels code. In
>>> this scenario, the bottleneck is actually within my FRP game loop and
>>> nothing to do with image loading! As a result, the profiles are fairly
>>> useless to me.
>>
>> While in this case the extra profiling data may be useless, it seems to me that in general you won't know a priori. I would prefer to have as much data available as possible, and then filter it.
>
> Agreed: I've often had to do a "cabal unpack <foo>", edit the .cabal
> file to add the profiling options and re-install that package (which
> is admittedly easier/nicer now with sandboxes).
Roman
Reply all
Reply to author
Forward
0 new messages