typed/racket + rackunit = trouble

[Anthony Carrico]

Unfortunately, rackunit doesn't mix that well with typed/racket (see
problem reports 15153, 15143). I guess this is because of the
typed/untyped boundary.

The good news is that it is fairly easy to work around the issues, for
example, you can do a check in the argument to check-true or check-false
so the computation stays in typed/racket.

I don't think I have an actual question for the list. I'm just
mentioning it here because I don't remember reading anywhere that
testing can be an issue in typed/racket. Discuss?

--
Anthony Carrico

[Lehi Toskin]

I've come across this before and I have mentioned this topic in IRC if not this list. There are a few macros in the rackunit that cannot and do not work well with typed/racket. The macros would have to be rewritten before they could play nice and as far as I know those aren't a priority.

[Raoul Duke]

(why are there 2 racke users groups? confusing.)

wait, who needs tests when you have static typing?! sheesh!!!!

:-)

[Sam Tobin-Hochstadt]

Unfortunately, the problem isn't just macros -- the underlying functions that actually implement RackUnit would have to be copied into Typed Racket. I don't know a way to make `check-eq?` work that doesn't require duplicating code. :(

Sam

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benjamin Greenman]

On Thu, Sep 17, 2015 at 4:38 PM, Raoul Duke <rao...@gmail.com> wrote:

wait, who needs tests when you have static typing?! sheesh!!!!

Really though, Typed Racket might benefit from a non-rackunit library that encourages a different style of testing.

(Instead of duplicating all the rackunit code.)

[Anthony Carrico]

On 09/17/2015 04:40 PM, Sam Tobin-Hochstadt wrote:
> Unfortunately, the problem isn't just macros -- the underlying functions
> that actually implement RackUnit would have to be copied into Typed
> Racket. I don't know a way to make `check-eq?` work that doesn't require
> duplicating code. :(

Is there a bigger problem with eq? and the type boundary? For example,
if I represent a "canonical" object with a reference type, could it
suddenly become non-canonical after crossing into untyped code? Or, if I
use it as a key in a weak hash table in TR, move it across the boundary
to untyped code, could the table lose that entry? I found this check-eq?
issue disturbing.

--
Anthony Carrico

[Sam Tobin-Hochstadt]

Yes, there is a larger issue here. In general, the typed/untyped boundary may change the eq? behavior. Some types won't (opaque types from the untyped side, monomorphic struct types from the typed side) but in general you won't get stable eq? results.

Sam

[Robby Findler]

I think that we need to work harder to deemphasize eq?, and warn
people that, when eq? returns #f, really you learned nothing (like you
need to pretend you just didn't even call eq?). If it returns #t, then
you learn something, of course.

Robby

[Neil Van Dyke]

Robby Findler wrote on 09/17/2015 09:18 PM:
> I think that we need to work harder to deemphasize eq?, and warn
> people that, when eq? returns #f, really you learned nothing (like you
> need to pretend you just didn't even call eq?). If it returns #t, then
> you learn something, of course.

What about `eq?` (and `memq`) of symbols?

[Robby Findler]

eq? on symbols is a special part of the specification and that seems
benign to me, all things considered. The "giant hash in the sky" that
makes sure that works isn't exactly trouble free, but we seem to have
it under control.

Robby

[Neil Van Dyke]

Whew. :) I only rarely have a non-symbol use for `eq?`, but I use `eq?`
heavily for symbols in everyday application code.

[Robby Findler]

FWIW, if you use equal? in those cases, you'll get the same
performance behavior and you will have fewer eq?s to audit when things
go wonky.

;)

Robby

[Sam Tobin-Hochstadt]

Unfortunately, that's only true when eq? produces #t, which probably isn't an issue when using eq? directly, but can be when using memq or similar. This benchmark suggests about a 10x speed difference when the symbols are different: http://pasterack.org/pastes/94877

Sam

[Neil Van Dyke]

Good to know. In that case, "just use `equal?`" is probably good advice
for new Racket people.

Personally, I'm super-comfortable with `eq?` for symbols (since, as you
know, it's a basic thing that's been used very heavily in Lisps for
old-school AI and compilers and such). So I might as well keep
efficient Scheme portability in this particular way, even were Racket to
officially guarantee that `equal?` is as fast.

[Robby Findler]

Sounds like something I would try to fix if I had time to really focus
on the eq? semantic question :(

[Sam Tobin-Hochstadt]

I don't know if it's really fixable -- equal? has to check that both arguments are symbols before producing #f, which is work that eq? doesn't do. Perhaps the difference can be reduced, though.

Sam

On Thu, Sep 17, 2015, 10:13 PM Robby Findler <ro...@eecs.northwestern.edu> wrote:

Sounds like something I would try to fix if I had time to really focus
on the eq? semantic question :(

[Robby Findler]

True.

Robby

On Thu, Sep 17, 2015 at 9:16 PM, Sam Tobin-Hochstadt

[Anthony Carrico]

This exchange fell off the list accidentally:

On Thu, Sep 17, 2015 at 8:58 PM, Anthony Carrico <acar...@memebeam.org>
wrote:

> On 09/17/2015 09:27 PM, Robby Findler wrote:
>> eq? on symbols is a special part of the specification and that seems
>> benign to me, all things considered. The "giant hash in the sky" that
>> makes sure that works isn't exactly trouble free, but we seem to have
>> it under control.
>

> Isn't eq? baked into the language already? Check-not-eq?, canonical
> objects (symbols are a special case of that), and weak tables are three
> places that the language is telling us we really /do/ learn something
> when eq? returns #f. I do understand why you would want deemphasize eq?,
> but I don't think you can get away with a just warning.

No, you're not learning something, even in that case. Well, I suppose
you might say that you're learning it didn't return #t, but what that
really means is "try harder". I consider check-eq? and friends a
mistake, symbols I've already mentioned and eq hash table are really
just a fancier form of eq? (so if I'm being careful I should include
them).

Really what I'm trying to say is that the language implementation
wants the freedom to adjust your program without having to be
constrained by eq tests that you might do. One example of this is
contracts. I might wish to be accept a function you give me, put a
contract on it, and give it back to you. This shouldn't really be
detectable if the contract doesn't fail. But it is, because of eq?.
Similarly, a compiler might want to change around exactly when it
allocates those cons cells (doing more sharing sometimes to reduce
memory footprint) but it can't because this is detectable via eq?, so
it isn't a behavior preserving transformation.

> As an example, I'm writing a toy sets-of-scopes expander. Scopes could
> be Natural, or whatever, but I'm wrapping them in a struct so I can use
> them to key weak tables. I can see other some other kind of language
> support as a replacement, like an Identity type, or some kind of Graph
> type where nodes not connected to a root are garbage (making local that
> formerly global feature of the heap).

I think weak tables are a separate issue. You mean eq tables, right?

And if so, you could have used a natural in a field and then used
equal? on that natural to get the same behavior, right? Then I can
decide to increment the natural when I want to and have precise
control over which ones are considered equal and which aren't. (I
agree this is not as convenient and, as I wrote earlier, alas, if i
were serious in fixing this, I would make that stuff more convenient.)

--
Anthony Carrico

[Raoul Duke]

It looks like it is a perennial truth of the universe that math is
hard. I don't know of a language ecosystem where "equals" is 'easy',
because it is subjective. Seems like offering different kinds of
equality testing is a reasonable approach. Having any kind of
preferred fundamental equality testing seems to have as many failures
and problems as it has uses.

[Anthony Carrico]

>> On 09/17/2015 09:27 PM, Robby Findler wrote:
> No, you're not learning something, even in that case. Well, I suppose
> you might say that you're learning it didn't return #t, but what that
> really means is "try harder". I consider check-eq? and friends a
> mistake, symbols I've already mentioned and eq hash table are really
> just a fancier form of eq? (so if I'm being careful I should include
> them).

I looked back at one of my bug reports, and indeed my example was using
an immutable object--my bad:

#lang typed/racket

(require typed/rackunit)
(define foo (cons 1 1))
(eq? foo foo)
(check-eq? foo foo)

But, here is an update:

#lang typed/racket

(require typed/rackunit)
(define foo (mcons 1 1))
(eq? foo foo)
(check-eq? foo foo)

And this is still broken.

> Really what I'm trying to say is that the language implementation
> wants the freedom to adjust your program without having to be
> constrained by eq tests that you might do.

I get that it wants the freedom, I'm just not sure it currently has the
freedom.

The docs say, "The eq? operator compares two values, returning #t when
the values refer to the same object. This form of equality is suitable
for comparing objects that support imperative update".

If this is no longer true, it implies that weak hash tables are broken
(they could arbitrarily drop entries), and that worries me.

Secondly, isn't equality the wrong word if a #t result means something
and #f doesn't? I suppose the issue is with the phrase "same object".

>> As an example, I'm writing a toy sets-of-scopes expander. Scopes could
>> be Natural, or whatever, but I'm wrapping them in a struct so I can use
>> them to key weak tables. I can see other some other kind of language
>> support as a replacement, like an Identity type, or some kind of Graph
>> type where nodes not connected to a root are garbage (making local that
>> formerly global feature of the heap).
>
> I think weak tables are a separate issue. You mean eq tables, right?

I did mean "make-weak-hash". It is desirable to couple the garbage
collection of binding table entries with the garbage collection of
scopes. I was using weak hash tables to map scope objects to binding
table partitions. mflatt achieves this by storing those partitions
directly inside the scope objects, avoiding a weak table. There are
costs and benefits of the two options.

Anyway, if we are allowed to use weak tables for memory management, then
we require a stable notion of identity. I agree that object identity is
ugly, so I'd trade it for an efficient, immutable, persistent, graph
datatype which allowed nodes to be collected when unreferenced from a
root :).

--
Anthony Carrico

[Anthony Carrico]

On 09/17/2015 11:03 PM, Anthony Carrico wrote:
> Really what I'm trying to say is that the language implementation
> wants the freedom to adjust your program without having to be
> constrained by eq tests that you might do. One example of this is
> contracts. I might wish to be accept a function you give me, put a
> contract on it, and give it back to you. This shouldn't really be
> detectable if the contract doesn't fail. But it is, because of eq?.
> Similarly, a compiler might want to change around exactly when it
> allocates those cons cells (doing more sharing sometimes to reduce
> memory footprint) but it can't because this is detectable via eq?, so
> it isn't a behavior preserving transformation.

Both these examples seem fine. Nobody expects procedures to be
comparable, and cons cells are immutable, so the docs say eq? isn't
suitable anyway (my bug report notwithstanding).

--
Anthony Carrico

[Anthony Carrico]

On 09/17/2015 11:03 PM, Anthony Carrico wrote:

> Really what I'm trying to say is that the language implementation
> wants the freedom to adjust your program without having to be
> constrained by eq tests that you might do. One example of this is
> contracts. I might wish to be accept a function you give me, put a
> contract on it, and give it back to you. This shouldn't really be
> detectable if the contract doesn't fail. But it is, because of eq?.
> Similarly, a compiler might want to change around exactly when it
> allocates those cons cells (doing more sharing sometimes to reduce
> memory footprint) but it can't because this is detectable via eq?, so
> it isn't a behavior preserving transformation.

[Robby Findler]

On Thu, Sep 17, 2015 at 11:24 PM, Anthony Carrico <acar...@memebeam.org> wrote:
> But, here is an update:
>
> #lang typed/racket
>
> (require typed/rackunit)
> (define foo (mcons 1 1))
> (eq? foo foo)
> (check-eq? foo foo)
>
> And this is still broken.

This might be a bug. I'll defer to the TR maintainers on that.

>> Really what I'm trying to say is that the language implementation
>> wants the freedom to adjust your program without having to be
>> constrained by eq tests that you might do.
>
> I get that it wants the freedom, I'm just not sure it currently has the
> freedom.

I believe it currently does not have that freedom. My messages were
meant to describe how I think eq? should be treated. The docs provide
stronger guarantees which, as you discovering, we don't quite live up
to. In particular, we don't currently have all of the implementation
technology we need to properly deny it that freedom. (Which sounds
strange, but we are balancing a lot of things here, and one of the is
TR's soundness, which currently has higher priority.)

> The docs say, "The eq? operator compares two values, returning #t when
> the values refer to the same object. This form of equality is suitable
> for comparing objects that support imperative update".
>
> If this is no longer true, it implies that weak hash tables are broken
> (they could arbitrarily drop entries), and that worries me.

I think this sentence is dodgy. It doesn't actually give you any
guarantees. The guarantee that you want is probably more of the form
"if there is only a single allocation point for a given object then
eq? should return #t when given it as both arguments" with some kind
of suitable definition of "allocation point" that doesn't include
flowing across a TR boundary. That's not precise either, but is
probably heading in the right direction.

> Secondly, isn't equality the wrong word if a #t result means something
> and #f doesn't?

This is the direction I would like to think about for eq?. That is, I
see eq? as having value for performance reasons, but for no other
reason. So, when you do an eq? test, you should always be
shortcircuiting some more complex computation. If you get a #f back,
then you are obligated to take the long way 'round, but if you get #t,
then you "learned something" that lets you avoid more computation.

And in an effort to be more clear: this is not currently what's
promised by Racket. This is more what I think we should try out and
see if we can make work.

>I suppose the issue is with the phrase "same object".



>>> As an example, I'm writing a toy sets-of-scopes expander. Scopes could
>>> be Natural, or whatever, but I'm wrapping them in a struct so I can use
>>> them to key weak tables. I can see other some other kind of language
>>> support as a replacement, like an Identity type, or some kind of Graph
>>> type where nodes not connected to a root are garbage (making local that
>>> formerly global feature of the heap).
>>
>> I think weak tables are a separate issue. You mean eq tables, right?
>
> I did mean "make-weak-hash". It is desirable to couple the garbage
> collection of binding table entries with the garbage collection of
> scopes. I was using weak hash tables to map scope objects to binding
> table partitions. mflatt achieves this by storing those partitions
> directly inside the scope objects, avoiding a weak table. There are
> costs and benefits of the two options.
>
> Anyway, if we are allowed to use weak tables for memory management, then
> we require a stable notion of identity. I agree that object identity is
> ugly, so I'd trade it for an efficient, immutable, persistent, graph
> datatype which allowed nodes to be collected when unreferenced from a
> root :).

That would be cool!

Robby

[Robby Findler]

I agree that an interesting other approach is to make eq? have some
well-defined behavior for mutable objects that amounts to "if I mutate
one, does the other one change". This will, however, mean that eq? is
not just a simple pointer equality check in the runtime system (well,
unless other changes are made in other parts of the runtime system
that I don't know how to do). So I don't think that's the best choice
(still making choices between things that are alternative futures for
Racket here, not talking about the current set of promises).

Robby

[Anthony Carrico]

On 09/18/2015 08:18 AM, Robby Findler wrote:
> On Thu, Sep 17, 2015 at 11:24 PM, Anthony Carrico <acar...@memebeam.org> wrote:
>> I get that it wants the freedom, I'm just not sure it currently has the
>> freedom.
>
> I believe it currently does not have that freedom. My messages were
> meant to describe how I think eq? should be treated.

Sounds like work, but I welcome our new equal? overlord.

--
Anthony Carrico

[Gustavo Massaccesi]

There is something very strange with memq vs member:
http://pasterack.org/pastes/80845 (note: I added one 0 to the loop
constants)

I used two mutable variables eq?? and equal?? to avoid (most of) the
optimizations and I get that the time with equal? is 2x than the time
with eq?. This is closer to the range I expected.

Using directly eq? and equal? is slightly faster, and the time is the
same because under the hood the optimizer transforms equal? to eq?
when one of the arguments is an explicit symbol.

The implementation of member and memq is very short but very
entangled. I think that with some tweaks is possible to reduce the
time difference to x2, and with some luck to make it more optimizer
friendly and gain some additional speed.

Gustavo

[Robby Findler]

that's great!

Robby

[Sam Tobin-Hochstadt]

When I run my original program on my own machine (instead of
Pasterack), the difference is more like 2-3x. So my original email
overstated the speed difference a bunch. I don't know what's going on
that makes it so different on pasterack.

An interesting side note -- we tried this program in Pycket, and two
interesting things happened:
- it turned out to be another test case for a problem we are working
on right now -- memq was too slow because it was used previously with
different types
- when that's fixed, memq and member are the same speed, because
Pycket's version of `eq?` is slower than it would be otherwise,
precisely because it has to maintain the semantics that Robby doesn't
like :)

Sam

On Sat, Sep 19, 2015 at 5:38 PM, Gustavo Massaccesi <gus...@oma.org.ar> wrote:
> There is something very strange with memq vs member:
> http://pasterack.org/pastes/80845 (note: I added one 0 to the loop
> constants)
>
> I used two mutable variables eq?? and equal?? to avoid (most of) the
> optimizations and I get that the time with equal? is 2x than the time
> with eq?. This is closer to the range I expected.
>
> Using directly eq? and equal? is slightly faster, and the time is the
> same because under the hood the optimizer transforms equal? to eq?
> when one of the arguments is an explicit symbol.
>
> The implementation of member and memq is very short but very
> entangled. I think that with some tweaks is possible to reduce the
> time difference to x2, and with some luck to make it more optimizer
> friendly and gain some additional speed.
>
> Gustavo
>
>
>

[Gustavo Massaccesi]

Probably the difference is caused by the diference of the Racket
version in your machine and in PasteRack. In my machine:

*** Using Racket 6.2
cpu time: 687 real time: 702 gc time: 0
cpu time: 14352 real time: 14838 gc time: 0
cpu time: 3619 real time: 3674 gc time: 0
cpu time: 5429 real time: 5430 gc time: 0
cpu time: 3245 real time: 3252 gc time: 0
cpu time: 3213 real time: 3242 gc time: 0

*** Using Racket almost the current HEAD (~one week old)
cpu time: 717 real time: 742 gc time: 0
cpu time: 4368 real time: 4465 gc time: 0
cpu time: 3526 real time: 3550 gc time: 0
cpu time: 5616 real time: 5809 gc time: 0
cpu time: 3057 real time: 3056 gc time: 0
cpu time: 2996 real time: 2996 gc time: 0

Gustavo


On Sat, Sep 19, 2015 at 9:22 PM, Sam Tobin-Hochstadt

[Sam Tobin-Hochstadt]

Impressive optimization! I wonder what it was that changed specifically.

Sam

On Sat, Sep 19, 2015 at 9:02 PM, Gustavo Massaccesi <gus...@oma.org.ar> wrote:
> Probably the difference is caused by the diference of the Racket
> version in your machine and in PasteRack. In my machine:
>
> *** Using Racket 6.2
> cpu time: 687 real time: 702 gc time: 0
> cpu time: 14352 real time: 14838 gc time: 0
> cpu time: 3619 real time: 3674 gc time: 0
> cpu time: 5429 real time: 5430 gc time: 0
> cpu time: 3245 real time: 3252 gc time: 0
> cpu time: 3213 real time: 3242 gc time: 0
>
> *** Using Racket almost the current HEAD (~one week old)
> cpu time: 717 real time: 742 gc time: 0
> cpu time: 4368 real time: 4465 gc time: 0
> cpu time: 3526 real time: 3550 gc time: 0
> cpu time: 5616 real time: 5809 gc time: 0
> cpu time: 3057 real time: 3056 gc time: 0
> cpu time: 2996 real time: 2996 gc time: 0
>
> Gustavo
>
>

> On Sat, Sep 19, 2015 at 9:22 PM, Sam Tobin-Hochstadt