Typed Racket and performance impact

[Paulo Matos]

Hi,

There was a time, many moons ago, not too far from the initial
implementations of typed racked when Sam was working on it that moving
from Racket to Typed Racket had a huge impact on performance.

Nowadays that might not be the case. I also noticed that using typed
racket might in itself improve performance (optimization coach seems to
mention Typed-Racket specific optimizations).

So, could anyone elaborate on their experiences to using typed racket
nowadays with regards to performance? Can it still be a step backwards
these days?

Kind regards,

--
Paulo Matos

[Matthias Felleisen]

This characterization is still good:

— fully typed programs might run significantly faster than untyped programs (30%)
— the performance of partially typed programs is all over the map (10% slower to 10000% slower)
— it remains difficult to predict whether the conversion of a single module introduces a serious slowdown
— slowdowns are caused by high frequency or large composite “crossings” of typed-untyped module boundaries
the allocation cost of wrappings of mutable and/or h-o values, and the wrappers themselves when these values are accessed.

If you want more specific information, we can post a journal submission that compares several Racket implementations (6.2, 6.3, and 6.4) on a range of programs using functional to ho object-oriented style.

— Matthias

[Paulo Matos]

On 19/02/18 13:18, Matthias Felleisen wrote:
>
> This characterization is still good:
>
> — fully typed programs might run significantly faster than untyped programs (30%)
> — the performance of partially typed programs is all over the map (10% slower to 10000% slower)
> — it remains difficult to predict whether the conversion of a single module introduces a serious slowdown
> — slowdowns are caused by high frequency or large composite “crossings” of typed-untyped module boundaries
> the allocation cost of wrappings of mutable and/or h-o values, and the wrappers themselves when these values are accessed.
>

Thanks, interesting to see that fully typed programs run faster though.
From your 4th point, one could assume that the more a program is typed,
the less crossing there will be and therefore the better the performance
will get?

> If you want more specific information, we can post a journal submission that compares several Racket implementations (6.2, 6.3, and 6.4) on a range of programs using functional to ho object-oriented style.
>

That would certainly be appreciated.

Thanks,
--
Paulo Matos

[Matthias Felleisen]

> On Feb 19, 2018, at 9:45 AM, Paulo Matos <pma...@linki.tools> wrote:
>
> Thanks, interesting to see that fully typed programs run faster though.

When I say “fully typed” you need to understand that some basic Racket
code may never be typed and is included here. I am referring to the code
that is under your control and I should add the adverb “usually” to this
sentence.

> From your 4th point, one could assume that the more a program is typed,
> the less crossing there will be and therefore the better the performance
> will get?

This is unfortunately not correct as stated. It depends on the order in which
you modify the modules. It will be rare that you pick a path along which every
step improves the performance after you picked the worst possible module to
equip with types. Indeed, I conjecture that such paths don’t exist. Asumu,
the original project lead, considered a similar question and I believe it
came to nothing.

>
>> If you want more specific information, we can post a journal submission that compares several Racket implementations (6.2, 6.3, and 6.4) on a range of programs using functional to ho object-oriented style.
>>
>
> That would certainly be appreciated.

I have temporarily posted the paper at the following URL:

http://www.ccs.neu.edu/home/matthias/Tmp/draft-performance-of-tr.pdf

Ben Greenman is the lead author and has taken this project under his wings. For any details on the measurements, he’s the man to talk to. In a sense, what we want is a re-evaluation of this benchmark suite for every release of Racket so that we know whether we are making progress. We just don’t have the manpower and machine resources to implement this idea.

— Matthias

[Ben Greenman]

Hi Paulo,

I hope you try using Typed Racket. If the performance turns out to be
a problem, we can try to help.


Here are some other experiences that people have shared on the list:

https://groups.google.com/d/msg/racket-users/rfM6koVbOS8/klVzjKJ9BgAJ

https://groups.google.com/d/msg/racket-users/6KQxpfMLTn0/lil_6qSMDAAJ

https://groups.google.com/d/msg/racket-users/oo_FQqGVdcI/p4-bqol5hV4J

https://groups.google.com/d/msg/racket-users/oo_FQqGVdcI/leUnIAn7yqwJ

[Paulo Matos]

On 19/02/18 17:10, Matthias Felleisen wrote:
>
>> On Feb 19, 2018, at 9:45 AM, Paulo Matos <pma...@linki.tools> wrote:
>>
>> Thanks, interesting to see that fully typed programs run faster though.
>
> When I say “fully typed” you need to understand that some basic Racket
> code may never be typed and is included here. I am referring to the code
> that is under your control and I should add the adverb “usually” to this
> sentence.
>

I understand.

>
>> From your 4th point, one could assume that the more a program is typed,
>> the less crossing there will be and therefore the better the performance
>> will get?
>
>
> This is unfortunately not correct as stated. It depends on the order in which
> you modify the modules. It will be rare that you pick a path along which every
> step improves the performance after you picked the worst possible module to
> equip with types. Indeed, I conjecture that such paths don’t exist. Asumu,
> the original project lead, considered a similar question and I believe it
> came to nothing.
>
>

But hopefully you'd get to a point (even if after some performance
worsening) where you'd get a benefit on the path to having your app
fully typed. Of course, as you said above if you have many crossings
into non typed code (libraries) then you might be in trouble. :-/

>
>>
>>> If you want more specific information, we can post a journal submission that compares several Racket implementations (6.2, 6.3, and 6.4) on a range of programs using functional to ho object-oriented style.
>>>
>>
>> That would certainly be appreciated.
>
>
>
> I have temporarily posted the paper at the following URL:
>
> http://www.ccs.neu.edu/home/matthias/Tmp/draft-performance-of-tr.pdf
>

Perfect. Got this.

> Ben Greenman is the lead author and has taken this project under his wings. For any details on the measurements, he’s the man to talk to. In a sense, what we want is a re-evaluation of this benchmark suite for every release of Racket so that we know whether we are making progress. We just don’t have the manpower and machine resources to implement this idea.
>

Thanks. Will contact him.

> — Matthias
>

--
Paulo Matos

[Paulo Matos]

Hi Ben,

Thanks for the shared links. That's very useful.
I am at a point where I feel I need typed racket and yet I am in the
world of squeezing performance wherever possible, therefore moving to
typed racket will have to be a process done in parallel to normal
product development. However, as I move from the untyped to the typed
world, it will be interesting to see how the benchmarks I have fluctuate
in time spent computing.

I might not be able to start this work before April/May time as I am in
the middle of a large delivery cycle but I will certainly get to it.

I will keep you posted.

--
Paulo Matos