backwards incompatibility in the handling of DrRacket's snip/values
38 views
Skip to first unread message
Robby Findler
May 8, 2015, 2:55:13 PM5/8/15
to d...@racket-lang.org, Christos Dimoulas, Scott Moore
I recently pushed a change to DrRacket in the way that it handles
snips that are the results of computations [1].
This change is backwards incompatible in the sense that code that
commonly worked before can commonly fail before. At a high-level that
code was buggy all along, but now the buggy behavior gets in the way
all the time instead of just in less common situations.
Let me give a little background to explain. The snip% class is how
racket/gui's editors represent things that can appear in an editor
(and be edited). The most common one of these is a string-snip%, which
represents a chunk of text (perhaps one line in a file that's being
edited in DrRacket). But there are lots of others, like image-snip%
that you get when you use the "Insert Image..." menu item or the
things you get when you evaluate 2/3 in the interactions window. There
are a bunch of more sophisticated such values (like plots) and the
thing they all have in common is that they are subclasses of snip%.
Turns out that DrRacket offering this interface to "fancy values" was
a bad choice, but it was one that I made a long time ago (late 90s or
so). So this message is about an attempt to fix it and the
repercussions.
The reason this was a bad choice is that the snip% API is all about
"things that live in editors". And one important feature of a thing
that lives in an editor is that you can save it. Editors edit files,
after all (duh!). Well, not all values are things that can easily be
saved in files (double duh!). This is the crux of the reason why it
was a bad choice.
This shows up in that many of these fancy value snips need procedures
that were compute random, unknown ahead of time things (think of a
plot of a function -- it plots an arbitrary function and it needs that
function when you zoom or whatever). So they just didn't implement the
marshalling part of the snip% interface.
Unfortunately, the change to DrRacket now requires that the
marshalling part of the interface to be implemented. Why? Because
moving a snip% object from the user's program's "space" (eventspace
specifically) to the DrRacket's only works by marshalling and
unmarshalling the snip. This triggers part of the snip API that
enables DrRacket to find and load it's implementation. And without
that step, the snip cannot actually be saved (PR 15049 [2]).
I'm writing to know what people think of this backwards incompatibility.
I don't know if it helps, but I'm willing to fix all of the existing
snips to use a new interface that's designed for "special values"
(which wouldn't ask the values to be able to be saved) or to actually
finish the implementation of snip%-ness, depending on which is
appropriate for each. The ones I know about are plot, pict3d, value
turtles, and frtime's signals. (If you know of more, please let me
know.)
I don't see a way to both fix PR 15049 and maintain backwards
compatibility, but it may be possible. (I'm confident it will be
complicated, if it is possible, however.)
And finally, this is also related to some security-based research work
that Scott and Christos are doing and I believe that the backwards
incompatible change is the most productive for that purpose, but my
guess is that maintaining backwards compatibility just makes some of
the hacking involved in that research more painful (as opposed to
impossible).
Anyway, if you made it this far in this message, thanks. Thoughts welcome.
Robby
[1]: https://github.com/racket/drracket/commit/01a19794ff9ab8188c0eb269b0ab12509096de1a
and https://github.com/racket/gui/commit/469add8d57c03f1ed269b48f5453dfad2f60d0c6
[2]: http://bugs.racket-lang.org/query/?cmd=view&pr=15049
snips that are the results of computations [1].
This change is backwards incompatible in the sense that code that
commonly worked before can commonly fail before. At a high-level that
code was buggy all along, but now the buggy behavior gets in the way
all the time instead of just in less common situations.
Let me give a little background to explain. The snip% class is how
racket/gui's editors represent things that can appear in an editor
(and be edited). The most common one of these is a string-snip%, which
represents a chunk of text (perhaps one line in a file that's being
edited in DrRacket). But there are lots of others, like image-snip%
that you get when you use the "Insert Image..." menu item or the
things you get when you evaluate 2/3 in the interactions window. There
are a bunch of more sophisticated such values (like plots) and the
thing they all have in common is that they are subclasses of snip%.
Turns out that DrRacket offering this interface to "fancy values" was
a bad choice, but it was one that I made a long time ago (late 90s or
so). So this message is about an attempt to fix it and the
repercussions.
The reason this was a bad choice is that the snip% API is all about
"things that live in editors". And one important feature of a thing
that lives in an editor is that you can save it. Editors edit files,
after all (duh!). Well, not all values are things that can easily be
saved in files (double duh!). This is the crux of the reason why it
was a bad choice.
This shows up in that many of these fancy value snips need procedures
that were compute random, unknown ahead of time things (think of a
plot of a function -- it plots an arbitrary function and it needs that
function when you zoom or whatever). So they just didn't implement the
marshalling part of the snip% interface.
Unfortunately, the change to DrRacket now requires that the
marshalling part of the interface to be implemented. Why? Because
moving a snip% object from the user's program's "space" (eventspace
specifically) to the DrRacket's only works by marshalling and
unmarshalling the snip. This triggers part of the snip API that
enables DrRacket to find and load it's implementation. And without
that step, the snip cannot actually be saved (PR 15049 [2]).
I'm writing to know what people think of this backwards incompatibility.
I don't know if it helps, but I'm willing to fix all of the existing
snips to use a new interface that's designed for "special values"
(which wouldn't ask the values to be able to be saved) or to actually
finish the implementation of snip%-ness, depending on which is
appropriate for each. The ones I know about are plot, pict3d, value
turtles, and frtime's signals. (If you know of more, please let me
know.)
I don't see a way to both fix PR 15049 and maintain backwards
compatibility, but it may be possible. (I'm confident it will be
complicated, if it is possible, however.)
And finally, this is also related to some security-based research work
that Scott and Christos are doing and I believe that the backwards
incompatible change is the most productive for that purpose, but my
guess is that maintaining backwards compatibility just makes some of
the hacking involved in that research more painful (as opposed to
impossible).
Anyway, if you made it this far in this message, thanks. Thoughts welcome.
Robby
[1]: https://github.com/racket/drracket/commit/01a19794ff9ab8188c0eb269b0ab12509096de1a
and https://github.com/racket/gui/commit/469add8d57c03f1ed269b48f5453dfad2f60d0c6
[2]: http://bugs.racket-lang.org/query/?cmd=view&pr=15049
Robby Findler
May 9, 2015, 12:24:36 PM5/9/15
to d...@racket-lang.org, Christos Dimoulas, Scott Moore
Another note I just realized: the backwards incompatible change also
fixed a problem with the way online check syntax expands some snips.
Before the change, if you were to copy and paste a fraction snip into
the definitions window, online check syntax would mumble something
about some interface that wasn't implemented and so it couldn't figure
out what program text that fraction was supposed to be. Well, it turns
out that that interface _was_ being implemented, but when you don't go
through the unmarshalling and marhsalling process, it was implementing
the wrong interface (I think that's what was wrong anyway): a
different instantiation of the modules leading to a different version
of the "same" interface. Now that the snip is reconstituted on the
DrRacket side, it implements the right interface.
Robby
fixed a problem with the way online check syntax expands some snips.
Before the change, if you were to copy and paste a fraction snip into
the definitions window, online check syntax would mumble something
about some interface that wasn't implemented and so it couldn't figure
out what program text that fraction was supposed to be. Well, it turns
out that that interface _was_ being implemented, but when you don't go
through the unmarshalling and marhsalling process, it was implementing
the wrong interface (I think that's what was wrong anyway): a
different instantiation of the modules leading to a different version
of the "same" interface. Now that the snip is reconstituted on the
DrRacket side, it implements the right interface.
Robby
Robby Findler
May 9, 2015, 1:05:20 PM5/9/15
to Jens Axel Søgaard, d...@racket-lang.org, Christos Dimoulas, Scott Moore
Does it make sense to extend record-dc% (or have another version of a
similar kind of class) that can record what happened to the cairo
context?
Robby
On Sat, May 9, 2015 at 11:37 AM, Jens Axel Søgaard
<jens...@soegaard.net> wrote:
> Consider a pict whose implementation take the underlying Cairo
> drawing context and draw on it using an external library via the FFI.
> Such a pict will not show up correctly in the DrRacket.
> [And it didn't before the recent change either. DrRacket used
> a recording-dc to copy the pict value, but record-dc% doesn't
> record any drawing done directly to the cairo context].
>
> The problem appears in the Racket bindings for Poppler,
> which renders pdf files to Cairo drawing contexts.
> A particular use case: Converting latex formulas into picts
> which can be used as labels in plot.
>
> https://github.com/soegaard/racket-poppler
>
> Is there a way to support this type of snip/pict too?
>
> /Jens Axel
>
>
>
>
similar kind of class) that can record what happened to the cairo
context?
Robby
On Sat, May 9, 2015 at 11:37 AM, Jens Axel Søgaard
<jens...@soegaard.net> wrote:
> 2015-05-08 20:55 GMT+02:00 Robby Findler <ro...@eecs.northwestern.edu>:
>>
>> I don't know if it helps, but I'm willing to fix all of the existing
>> snips to use a new interface that's designed for "special values"
>> (which wouldn't ask the values to be able to be saved) or to actually
>> finish the implementation of snip%-ness, depending on which is
>> appropriate for each. The ones I know about are plot, pict3d, value
>> turtles, and frtime's signals. (If you know of more, please let me
>> know.)
>
>
> A related problem: picts are displayed in DrRacket via snips.
>>
>> I don't know if it helps, but I'm willing to fix all of the existing
>> snips to use a new interface that's designed for "special values"
>> (which wouldn't ask the values to be able to be saved) or to actually
>> finish the implementation of snip%-ness, depending on which is
>> appropriate for each. The ones I know about are plot, pict3d, value
>> turtles, and frtime's signals. (If you know of more, please let me
>> know.)
>
>
> Consider a pict whose implementation take the underlying Cairo
> drawing context and draw on it using an external library via the FFI.
> Such a pict will not show up correctly in the DrRacket.
> [And it didn't before the recent change either. DrRacket used
> a recording-dc to copy the pict value, but record-dc% doesn't
> record any drawing done directly to the cairo context].
>
> The problem appears in the Racket bindings for Poppler,
> which renders pdf files to Cairo drawing contexts.
> A particular use case: Converting latex formulas into picts
> which can be used as labels in plot.
>
> https://github.com/soegaard/racket-poppler
>
> Is there a way to support this type of snip/pict too?
>
> /Jens Axel
>
>
>
>
Jens Axel Søgaard
May 12, 2015, 2:04:55 AM5/12/15
to Robby Findler, d...@racket-lang.org, Christos Dimoulas, Scott Moore
2015-05-08 20:55 GMT+02:00 Robby Findler <ro...@eecs.northwestern.edu>:
I don't know if it helps, but I'm willing to fix all of the existing
snips to use a new interface that's designed for "special values"
(which wouldn't ask the values to be able to be saved) or to actually
finish the implementation of snip%-ness, depending on which is
appropriate for each. The ones I know about are plot, pict3d, value
turtles, and frtime's signals. (If you know of more, please let me
know.)
Jens Axel Søgaard
May 12, 2015, 2:04:55 AM5/12/15
to Robby Findler, d...@racket-lang.org, Christos Dimoulas, Scott Moore
2015-05-09 19:05 GMT+02:00 Robby Findler <ro...@eecs.northwestern.edu>:
Does it make sense to extend record-dc% (or have another version of a
similar kind of class) that can record what happened to the cairo
context?
Yes, I think that would do it.
FWIW:
/Jens Axel
Reply all
Reply to author
Forward
0 new messages