Protocol Language | Codec

[sirinath]

Hello,

Many computing systems need to communicate through different protocols. Hence is it possible to have a DSL to create encode and decode binary formats and enforce protocol communication which would be statically verified.

Suminda

[Stephen De Gabrielle]

I'm sure this has been discussed before

I did a quick search of racket-users via the google groups web interface and it seemed to come up with some promising results:

https://groups.google.com/forum/m/?hl=en#!searchin/racket-users/Binary

I also searched for ASN.1 but had less luck.

Good luck

--
You received this message because you are subscribed to the Google Groups "Racket Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+...@googlegroups.com.
To post to this group, send email to racke...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/dbde3c27-acac-4032-868b-48e6b32a91c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kind regards,
Stephen
--

[Neil Van Dyke]

Yes, it is possible. The two publicly-available related packages for
the binary parsing parts of protocl I recall:

* Tony Garnock-Jones's `bitsyntax`:
https://github.com/tonyg/racket-bitsyntax

* Roman Klochkov's `binary-class`:
https://docs.racket-lang.org/binary-class/index.html
https://groups.google.com/forum/#!msg/racket-users/hx71t8kgnx0/QwBFtLKvBxoJ

If anyone wants to work more on binary parsing/writing
DSLs/mini-languages, two popular formats to test your abstractions might
be TIFF and EXIF (you need them for JPEG file metadata). And a media
container format like Ogg or MP4. And IP packets, and the encoding of
TCP. The Bittorrent format might be easy to start with. At some point,
take a look at Google protocol buffers (though you'd want a separate
language for that, perhaps that expands to your more general binary
language). You'll often have to deal with things like reading a small
number of bytes that signal how to parse some of the immediately
following bytes (like a type tag or a length). Some, like parsing EXIF
metadata, require reading numbers that are like memory addresses or
offsets in the input or part of the input, and then
dereferencing/jumping&returning. Sometimes you'll have pairs or
tuples. What they usually *don't* require is a parser with more than 1
token lookahead or backtracking, maybe because these binary formats are
usually designed for fast machine processing rather than to be
human-readable. Handwritten code for parsing some of these is in
"http://www.neilvandyke.org/racket/mediafile/".

Once you get beyond parsing real-world protocol binary formats, the
other aspects of protocols get more complicated, and I think you'll
often need a general-purpose (or at least Turing-complete) language.
However, for a particular protocol, or class of protocols you identify,
you might be able to come up with a DSL that expresses everything, and
which is easier to verify. You might also have a DSL layer that almost
exactly matches the formal specification for the protocol (though it's
usually not that easy, and specs usually get fuzzy on important details).

Oh yeah, if you're working on high-performance binary parsing, consider
having your minilanguage expand to code that avoids copying or
allocating new objects. So, say, instead of the usual reading from an
input port and returning values it allocates, it reads (perhaps from a
Racket `bytes?`) and performs imperative actions that the programmer has
specified in terms of the minilanguage, perhaps as a fold. For folding
examples, see Oleg Kiselyov's SSAX or
"http://www.neilvandyke.org/racket/json-parsing/". And if you find
yourself considering using `call/cc`, strongly consider using
multithreading or something else instead.

[George Neuner]

On Mon, 13 Feb 2017 06:10:34 -0500, Neil Van Dyke
<ne...@neilvandyke.org> wrote:

>Once you get beyond parsing real-world protocol binary formats, the
>other aspects of protocols get more complicated, and I think you'll
>often need a general-purpose (or at least Turing-complete) language.

Most real-world communication protocols can be implemented using
[relatively] simple state machines. There is no question that a GPL
may be needed to handle the content of "messages", but the actual
processing to implement the protocol rarely *requires* that kind of
power.

Obviously, the choice of language affects ease of implementation.

YMMV.

>However, for a particular protocol, or class of protocols you identify,
>you might be able to come up with a DSL that expresses everything, and
>which is easier to verify. You might also have a DSL layer that almost
>exactly matches the formal specification for the protocol (though it's
>usually not that easy, and specs usually get fuzzy on important details).

A generic state machine generator / combinator might be a useful
starting point. Is there anything like that available for Racket?

George

[Neil Van Dyke]

George Neuner wrote on 02/13/2017 01:39 PM:
>
>> However, for a particular protocol, or class of protocols you identify,
>> you might be able to come up with a DSL that expresses everything, and
>> which is easier to verify. You might also have a DSL layer that almost
>> exactly matches the formal specification for the protocol (though it's
>> usually not that easy, and specs usually get fuzzy on important details).
> A generic state machine generator / combinator might be a useful
> starting point. Is there anything like that available for Racket?

I don't know of an off-the-shelf reusable one. Implementing a simple
FSM in a Scheme descendant is often (usually?) best a call graph of
state procedures, each with `case` or other branching on event, and
doing tail calls to transition.

Shriram did use a simple FSM as an example for an introduction to macros:
https://cs.brown.edu/~sk/Publications/Papers/Published/sk-automata-macros/paper.pdf

In the early days of Java, I made a DSL with rich state-ish dynamic
modeling semantics, which generated Java code, and I included an example
of TCP connection behavior, but I never used it for real-world
problems. I've been wanting to do more on this, as a Racket syntax
extension suitable for real-world high-performance work, but not yet had
an excuse that simultaneously paid bills. :)

[Jay McCarthy]

On Mon, Feb 13, 2017 at 1:39 PM, George Neuner <gneu...@comcast.net> wrote:

A generic state machine generator / combinator might be a useful
starting point.  Is there anything like that available for Racket?

http://docs.racket-lang.org/automata/ 

--

-=[     Jay McCarthy               http://jeapostrophe.github.io    ]=-

-=[ Associate Professor        PLT @ CS @ UMass Lowell     ]=-

-=[ Moses 1:33: And worlds without number have I created; ]=-