binary protocols code generators (parser/writer/validator/[de]serializer)

[Dmitry Ponyatov]

Are there any tools to implement code generators for binary protocols parsers and writers? 

I mean something like lex/yacc but for binary protocols (binary file formats, arbitrary net protocols, serialization for complex APIs etc).

[Dmitry Ponyatov]

• https://www.protlr.com/example
  
• https://kaitai.io
  
• http://www.icsi.berkeley.edu/pubs/networking/binpacIMC06.pdf
  

What else with good tutorials included?

Arbitrary bitwidth fields and endianness support are required.

The first application candidates are 

• full-featured ASN.1 parser/generator/decompiler modules and some variants (especially SMTP etc).
  
• process any file formats has a specification (images, office, CAD,...) 

PS: variative parsing like DCG implementation in golang is also very interesting

PSS: generic tutorial on writing binary parsers is also very interesting for my metaprogramming working

[Ronny Bangsund]

Protocol Buffers springs to mind. It's a compact binary protocol with predefined field order, allowing you to skip unused fields and maintain backwards compatibility. You could probably use it as a storage format too. It's not in the Lex/Yacc territory, but it DOES generate all the code you need to handle the defined structures.

Protobuf is the message definition, and it goes hand in hand with gRPC to generate client code plus server code stubs for secure network communication with optional streaming:

https://developers.google.com/protocol-buffers/

https://grpc.io

If you're looking for tools for existing binary formats, I dunno. Protobuf would only be halfway there, as I don't think it supports unions with overlapping fields as you could find in some (like the PE binary format for Win32/Win64).

[Dmitry Ponyatov]

Protocol Buffers springs to mind.

Totally wrong. Protocol Buffers unable to parse TIFF or STEP file.

 

[Robert Engels]

He stated that protobufs would only be 50% for existing formats... no need to pounce. 

On Jan 14, 2019, at 4:04 AM, Dmitry Ponyatov <dpon...@gmail.com> wrote:

Protocol Buffers springs to mind.

Totally wrong. Protocol Buffers unable to parse TIFF or STEP file.

 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.