Request: ability to use bigarray instead of string

[Jessica Hamilton]

Hi,

I'd like to request the ability to use bigarray instead of string.

Given API access to get an element is very similar between strings and
bigarrays, do you think adapting the preprocessor would be difficult?

You could have a sub-module: Bitstring.Bigarray.

For ease of compatibility between String and Bigarray, the type of the
bigarray should probably be (int/char, int8_unsigned_elt, c_layout)
Array1.t. That way, all pre-existing math is unaltered too. int would
probably be better to avoid having to convert to an int, which you
must already do anyway.

Does this sound feasible?

Most of my uses of Bitstring would require conversions between
bigarrays and strings, so such a feature would be most appreciated!

Many Thanks,

Jessica

[Richard Jones]

I'm interested to know why bigarrays? Is it just because you've
already got the data in this format, or are you on a 32 bit platform
and hitting the rather small limit on string sizes?

Anyhow, such a change would be substantial. You are welcome to
explore patches which do this ...

Rich.

--
Richard Jones
Red Hat

[Jessica Hamilton]

It's pretty much because the data is already in this format.

This is for my operating system project :) I already use your
bitstring stuff for protocol parsing, but yet to test it further --
have yet to plug protocol parsing to the network driver, with a
netstack currently being put together.

I could probably change everything from bigarrays to strings. Maybe
it'd be faster. Given similar APIs, it's probably a relatively
straightforward change.

I was mainly thinking ahead to my goal of pulling wave files over
network and sending to the audio driver, which already works with
bigarrays. I'll try strings instead.

Jessica

> Red Hat- Hide quoted text -
>
> - Show quoted text -

[Jessica Hamilton]

Heh, I just remembered why I used bigarrays now. I'm running in 32-
bit, and a 60MB wave file doesn't fit into an ocaml string easily...
Sys.max_string_length being 16,777,211.

What ever happened to the streaming API? Is this still under
development or anything? This would work around string length
limitations too :)

Jessica

On May 9, 8:01 pm, Jessica Hamilton <jessica.l.hamil...@gmail.com>
wrote:

> > - Show quoted text -- Hide quoted text -

[Romain Beauxis]

Hi all !

Le Saturday 09 May 2009 10:07:35 Jessica Hamilton, vous avez écrit :
> What ever happened to the streaming API? Is this still under
> development or anything? This would work around string length
> limitations too :)

I've always wanted to have such an API. I was wondering whether this would be
possible using some abstract String-compatible module, which would abstract
all operations on strings.

Using a syntactic sugar provided by camlp4, it would even be possible to use
the same code for bitstring..

Romain

[Richard Jones]

On Sat, May 09, 2009 at 01:07:35AM -0700, Jessica Hamilton wrote:
> What ever happened to the streaming API? Is this still under
> development or anything? This would work around string length
> limitations too :)

Well the streaming API is still waiting for someone to do the work,
_and_ no one really knows how it should look and still maintain some
efficiency and not multiply the current codebase by a factor of 2.
That's the bottom line with streaming ...

You might want to talk to the Liquid Soap developers who have
expressed some interest in this too.

http://savonet.sourceforge.net/

[Jessica Hamilton]

So I had a look at your pa_bitstring file :)

It's surprising how little code in pa_bitstring actually depends on
the underlying datatype at all. I haven't attempted to do bitstring
construction with bigarray yet (I deleted it all for sake of testing),
and I actually got it compiled using bigarrays :) Although I didn't
have utility functions to do the conversion between bigarrays and
strings (have something for that now, but they're not standard
bigarray functions), which kind of stopped further exploration of the
approach.

Given the lack of bigarray <-> string conversions, it's probably not
possible to provide a patch, without patching the Bigarray library =/

Was a fun little foray into bitstring internals though :)

Jessica

On May 9, 7:52 pm, Richard Jones <r...@annexia.org> wrote:

[Richard Jones]

On Wed, May 13, 2009 at 09:59:41PM -0700, Jessica Hamilton wrote:
> Given the lack of bigarray <-> string conversions, it's probably not
> possible to provide a patch, without patching the Bigarray library =/

We can't just provide these conversion functions in bitstring
itself? Or do these functions depend in some fundamental way
on the internals of bitstring?