custom byte encoders?

41 views
Skip to first unread message

je...@lisp.sh

unread,
Apr 23, 2021, 1:01:45 AM4/23/21
to Racket Users
Is it possible to make your own byte encoder? Section 4.5.4 of the reference talks about them and describes the handful of built-in ones. I don't see a way of making my own byte encoders, though. I'd like to have my own way of validating bytes and performing replacements, similar to what is done with the "UTF-8-permissive" encoder (for example, certain junk characters get replaced by #\uFFFD), but I'd like to do a bit more than that. For example, I'd like replace some exotic but otherwise valid UTF-8 characters with #\uFFFD. It seems that the only way to do that is to write my own custom input (or output) port, with the validation/replacement logic contained within the custom port's peek/read-bytes/write-bytes procedures. Or perhaps I overlooking something?

Jesse

je...@lisp.sh

unread,
Apr 23, 2021, 3:20:43 AM4/23/21
to Racket Users
The more I read in section 4.5.4 the more it dawns on me that what I'm after may not be possible without (a) a custom patched version of Racket and/or (b) a custom patched version of libiconv. An alternative -- I think -- would be a custom input port, but even that might require (a) and/or (b) above. Naturally, I'd prefer not to have to have a private variant of Racket, and I'd rather not roll my own encoder in C. If there's no other way, I'll do it. But maybe I'm overlooking something simpler.

Matthew Flatt

unread,
Apr 23, 2021, 8:53:49 AM4/23/21
to je...@lisp.sh, Racket Users
A new custom port sounds like the right idea. The `reencode-input-port`
and ``reencode-output-port` implementations in
"collects/racket/port.rkt" are tied to byte converters, but they should
be the right idea and could be generalized to other conversion
functions. You might have to copy them for now, but you could propose
changes so that future Racket versions would have the right
abstraction.

Matthew
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to racket-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/2a29be0f-382d-44ee-b691-91c6b527f
> c48n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages