[Haskell-cafe] Backtracking when building a bytestring

[David Turner]

Hi,

I'm working on a network protocol that involves sending frames of data prefixed by their length and a checksum. The only realistic way to work out the length of a frame is to actually write the bytes out, although the length and checksum take up a fixed number of bytes.

If I were working in C I'd be filling a buffer, leaving space for the length/checksum, and then go back and fill them in at the end. So this is _almost_ what ByteString.Builder does except for the backtracking bit.

Does anyone know if there's an implementation of a thing that's a bit like ByteString.Builder but also allows for this kind of backtracking? Ideally I want to be able to batch up a number of frames into a single buffer, and deal gracefully with overflowing the buffer by allocating some more, much as Builder does.

I can't think of a terribly good way of doing this using the existing Builder implementation as it stands, although it looks quite easy to modify it to add this functionality so I might just do that locally if needs be.

Cheers,

David

[Axel Mannhardt]

Hi,

On Fr, 2016-06-24 at 10:23 +0100, David Turner wrote:
> I'm working on a network protocol that involves sending frames of data
> prefixed by their length and a checksum. The only realistic way to work out
> the length of a frame is to actually write the bytes out, although the
> length and checksum take up a fixed number of bytes.

this might be of interest to you - the explanation is in German though:

http://nfa.imn.htwk-leipzig.de/HAL2015/programm/slides/breitner.pdf

Cheers,
Axel

--
Axel Mannhardt
Master of Science (M.Sc.)

freiheit.com technologies gmbh
Budapester Straße 45
20359 Hamburg / Germany
fon +49 40 / 890584-0
fax +49 40 / 890584-20
Hamburg HRB 70814

https://twitter.com/freiheit_com

35AB AE2D 4002 DE31 C7E2 CC21 4D3F 9E9C 5EE9 B3B3
Geschäftsführer: Claudia Dietze, Stefan Richter

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

[Yuras Shumovich]

I think it is possible (and easy) to implement with
Data.ByteString.Builder. Just use the low-level interface: http://hacka
ge.haskell.org/package/bytestring-0.10.8.1/docs/Data-ByteString-
Builder-Extra.html#v:runBuilder

BufferWriter takes a buffer to write to and returns a continuation to
be called in case the buffer is too small. The idea is to allocate a
buffer, but reserve necessary space for length and checksum. Then run
the builder, get the resulting ByteString (or work directly with `Ptr
Word8`), calculate length and check sum and write then into the
reserved space.

Just be careful when using the low level API, and read the docs
carefully.

Thanks,
Yuras.

[David Turner]

On 24 June 2016 at 11:41, Yuras Shumovich <shumo...@gmail.com> wrote:

I think it is possible (and easy) to implement with
Data.ByteString.Builder. Just use the low-level interface: http://hacka
ge.haskell.org/package/bytestring-0.10.8.1/docs/Data-ByteString-
Builder-Extra.html#v:runBuilder

BufferWriter takes a buffer to write to and returns a continuation to
be called in case the buffer is too small. The idea is to allocate a
buffer, but reserve necessary space for length and checksum. Then run
the builder, get the resulting ByteString (or work directly with `Ptr
Word8`), calculate length and check sum and write then into the
reserved space.

Just be careful when using the low level API, and read the docs
carefully.

Aha, thanks, that looks like just what I'm after! Great.

On 24 June 2016 at 10:39, Axel Mannhardt <axel.ma...@freiheit.com> wrote:

this might be of interest to you - the explanation is in German though:

http://nfa.imn.htwk-leipzig.de/HAL2015/programm/slides/breitner.pdf

Cheers,
Axel

My German is nonexistent but fortunately the code is English enough to read, thanks. Unfortunately,

> type ByteString = [Word8]

This is not the ByteString I'm looking for!

Cheers,

David