Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Scala ByteString
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  13 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Richard Wallace  
View profile  
 More options Aug 4 2012, 10:41 pm
From: Richard Wallace <rwall...@thewallacepack.net>
Date: Sat, 4 Aug 2012 19:41:23 -0700
Local: Sat, Aug 4 2012 10:41 pm
Subject: Scala ByteString
Hey everybody,

I just pushed an initial pass at a ByteString library for Scala.

A ByteString can be created from an Array[Byte] using ByteString.pack.
 There are also some functions for reading from InputStreams and
writing to OutputStreams in purefn.bytestring.io and pimps in
purefn.bytestring.syntax.

A  ByteString is backed by a java.nio.ByteBuffer and is immutable.
Currently the ByteBuffers are always array based.  I'd like to do some
comparing the performance of array based buffers to direct buffers and
may change it depending on the results.  At the very least, there will
probably be specialized functions for reading/writing using direct or
array based buffers so that you can choose which you want to use.

For now, I just wanted to get _something_ done and I'm pretty happy
with the API so far.

In the next week or so I plan to add tests, enumerator/iteratee
support and ByteChannel support.

I'd love feedback.

Rich


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Roland Kuhn  
View profile  
 More options Aug 6 2012, 2:19 am
From: Roland Kuhn <goo...@rkuhn.info>
Date: Mon, 6 Aug 2012 08:19:09 +0200
Local: Mon, Aug 6 2012 2:19 am
Subject: Re: [scalaz] Scala ByteString
Hi Richard,

have you had a look at https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/ak... ? It will enter the Scala distribution by way of akka-actor.jar for 2.10 and it would be awesome if we could make it so good that only one implementation is needed (plus a few type classes in scalaz, I guess).

Regards,

Roland Kuhn
Typesafe — The software stack for applications that scale
twitter: @rolandkuhn

On 5 aug 2012, at 04:41, Richard Wallace <rwall...@thewallacepack.net> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Richard Wallace  
View profile  
 More options Aug 6 2012, 2:24 am
From: Richard Wallace <rwall...@thewallacepack.net>
Date: Sun, 5 Aug 2012 23:24:02 -0700
Local: Mon, Aug 6 2012 2:24 am
Subject: Re: [scalaz] Scala ByteString

By "Scala distribution" do you mean the standard library?
On Aug 5, 2012 11:19 PM, "Roland Kuhn" <goo...@rkuhn.info> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Roland Kuhn  
View profile  
 More options Aug 6 2012, 2:38 am
From: Roland Kuhn <goo...@rkuhn.info>
Date: Mon, 6 Aug 2012 08:38:23 +0200
Local: Mon, Aug 6 2012 2:38 am
Subject: Re: [scalaz] Scala ByteString

As I linked to, it is still in the akka source repository: the scala distribution for 2.10 will consist of multiple JARs, e.g. scala-library, scala-reflect, scala-actors, akka-actors, … where some of them are built from the scala/scala repository and others are not. This is my complicated way of saying that I would call scala-library.jar the “standard library”, but that is my personal view.

So, if we can arrive at a common and high performance ByteString implementation, then I think it should be moved into the standard library proper, under the scala.collection package. I would presume that such an addition could happen in a minor release.

Regards,

Roland

6 aug 2012 kl. 08:24 skrev Richard Wallace:

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Richard Wallace  
View profile  
 More options Aug 6 2012, 3:12 am
From: Richard Wallace <rwallace1...@gmail.com>
Date: Mon, 6 Aug 2012 00:12:35 -0700
Local: Mon, Aug 6 2012 3:12 am
Subject: Re: [scalaz] Scala ByteString
The reason I asked is because the standard lib often seems to be where
things go to die.  Not just in Scala, but I've seen it in other
languages too.  I'd just as soon keep useful things out of the
language stdlib so that they can be more rapidly improved independent
of the rest of the language.

I'm also not entirely sure that being a part of another library is the
best place for something like this, especially something as large as
Akka.  There is something to be said for small libraries that can be
quickly evolve.

All that being said, I'd be totally up for creating a common
ByteString that is really really fast.  We will probably never get it
quite as good as the Haskell version because we lack array fusion and
some low level memory functions (realloc, memchr, and a few others I
can't think of right now) - but I've been very seriously considering
writing some JNI to fix that problem.  Array fusion is the biggest
win, and maybe macros could help make that happen (Paul P posted
something that looked interesting in this regard a while back, but
with very few details).


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tony Morris  
View profile  
 More options Aug 6 2012, 3:23 am
From: Tony Morris <tonymor...@gmail.com>
Date: Mon, 6 Aug 2012 17:23:49 +1000
Local: Mon, Aug 6 2012 3:23 am
Subject: Re: [scalaz] Scala ByteString

I totally agree that if it useful, it belongs. All you need is three or so
instances and a few handy combinators to compel the case.

It is a shame that it is String which gets special language syntax support
in Scala (and only slightly better in Haskell with [Char] mod
OverloadedStrings). I totally do not buy the "yeah but Java interop" lame
excuse.

Assuming Cord is a reasonable standard for a general purpose String and it
has the library support to go with it, then I think IsString would be great
to give library users their best choice. i.e. I would love to defer the
choice of string with a generalisation and let users decide if they will
use String at their own peril or perhaps Cord or ByteString.Char if they
wish to forego the syntax support in exchange for a useful string.
On Aug 6, 2012 5:12 PM, "Richard Wallace" <rwallace1...@gmail.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Roland Kuhn  
View profile  
 More options Aug 6 2012, 3:25 am
From: Roland Kuhn <goo...@rkuhn.info>
Date: Mon, 6 Aug 2012 09:25:17 +0200
Local: Mon, Aug 6 2012 3:25 am
Subject: Re: [scalaz] Scala ByteString
I see what you’re getting at, and in general I find myself in resonance, but in this specific case I’d beg to differ: ByteString is a very stable problem, with all properties known up-front, which means that my idea would be to write a good solution once and then be done with it. The “quickly evolving” stage surely comes long before the “consider putting into standard library” stage. And after that, it will be pretty good by definition, and if a 2% performance improvement can be achieved, that’s fine, but that can then also wait some time for the next update release.

If you want to be sufficiently fanatic (i.e. go down the JNI route), then it would need to be a separate library in any case, since JNI is just horrible to support in general.

Which brings my to the conclusion that we might be aiming for different things after all: I’m shooting for a solid and fast implementation using only JVM-bytecode, so that it can enter the standard library proper, while you seem to be going for the fastest possible at all costs.

6 aug 2012 kl. 09:12 skrev Richard Wallace:

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jason Zaugg  
View profile  
 More options Aug 6 2012, 3:26 am
From: Jason Zaugg <jza...@gmail.com>
Date: Mon, 6 Aug 2012 09:26:30 +0200
Local: Mon, Aug 6 2012 3:26 am
Subject: Re: [scalaz] Scala ByteString

On Mon, Aug 6, 2012 at 9:23 AM, Tony Morris <tonymor...@gmail.com> wrote:
> I totally agree that if it useful, it belongs. All you need is three or so
> instances and a few handy combinators to compel the case.

> It is a shame that it is String which gets special language syntax support
> in Scala (and only slightly better in Haskell with [Char] mod
> OverloadedStrings). I totally do not buy the "yeah but Java interop" lame
> excuse.

Actually with a macro you could convert a string literal to another
representation at compile time.

-jason


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Roland Kuhn  
View profile  
 More options Aug 6 2012, 3:30 am
From: Roland Kuhn <goo...@rkuhn.info>
Date: Mon, 6 Aug 2012 09:30:11 +0200
Local: Mon, Aug 6 2012 3:30 am
Subject: Re: [scalaz] Scala ByteString

6 aug 2012 kl. 09:26 skrev Jason Zaugg:

> On Mon, Aug 6, 2012 at 9:23 AM, Tony Morris <tonymor...@gmail.com> wrote:
>> I totally agree that if it useful, it belongs. All you need is three or so
>> instances and a few handy combinators to compel the case.

>> It is a shame that it is String which gets special language syntax support
>> in Scala (and only slightly better in Haskell with [Char] mod
>> OverloadedStrings). I totally do not buy the "yeah but Java interop" lame
>> excuse.

> Actually with a macro you could convert a string literal to another
> representation at compile time.

Since I’m not aware of array literals at the JVM level, how much would you gain? The other representation would have to result in byte-code which reconstructs the same thing at runtime.

Regards,

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tony Morris  
View profile  
 More options Aug 6 2012, 3:34 am
From: Tony Morris <tonymor...@gmail.com>
Date: Mon, 06 Aug 2012 17:34:34 +1000
Local: Mon, Aug 6 2012 3:34 am
Subject: Re: [scalaz] Scala ByteString
On 06/08/12 17:26, Jason Zaugg wrote:
> On Mon, Aug 6, 2012 at 9:23 AM, Tony Morris <tonymor...@gmail.com> wrote:
>> I totally agree that if it useful, it belongs. All you need is three or so
>> instances and a few handy combinators to compel the case.

>> It is a shame that it is String which gets special language syntax support
>> in Scala (and only slightly better in Haskell with [Char] mod
>> OverloadedStrings). I totally do not buy the "yeah but Java interop" lame
>> excuse.
> Actually with a macro you could convert a string literal to another
> representation at compile time.

> -jason

Well that would be even more awesome if we could keep it as flexible as
a type-class but with all the other benefits of macros.

--
Tony Morris
http://tmorris.net/


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Richard Wallace  
View profile  
 More options Aug 6 2012, 4:01 am
From: Richard Wallace <rwall...@thewallacepack.net>
Date: Mon, 6 Aug 2012 01:01:37 -0700
Local: Mon, Aug 6 2012 4:01 am
Subject: Re: [scalaz] Scala ByteString
I'd like to get as fast as possible before going down the JNI path
because I agree that it would be a PITA to support.  I also agree that
_eventually_ it would make sense to make this part of the standard
lib, once things stabilize.  At the same time, I think it makes sense
for a ByteString library to be it's own library, not a smaller part of
something larger.

TBH I have no idea how performant by current implementation of
ByteString measures up.  Like I said before, I just wanted to get
_something_ up.  I'm writing tests now and then I plan on doing some
performance testing and trying out some different implementations.
When I started I went down a similar path as Runàr's Cord
implementation, using a Rope[Byte].  I ran into some problems with
that and thought that ByteBuffers would make more sense because of the
IO operations involved.  Looking at it again, I may change the impl to
use Rope[ByteBuffer] since I couldn't make use of memchr or other low
level functions that I was hoping to be to use.  But before I do
something like that I would very much like to get some performance
numbers.

Rich


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Tony Morris  
View profile  
 More options Aug 6 2012, 4:07 am
From: Tony Morris <tonymor...@gmail.com>
Date: Mon, 06 Aug 2012 18:07:29 +1000
Local: Mon, Aug 6 2012 4:07 am
Subject: Re: [scalaz] Scala ByteString
I don't know if you have ever supported a JNI library, but I have at
(IBM's rough equivalent to keytool) and I must caution you, it becomes a
full-time job.

On 06/08/12 18:01, Richard Wallace wrote:

--
Tony Morris
http://tmorris.net/

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Richard Wallace  
View profile  
 More options Aug 6 2012, 4:36 am
From: Richard Wallace <rwall...@thewallacepack.net>
Date: Mon, 6 Aug 2012 01:36:45 -0700
Local: Mon, Aug 6 2012 4:36 am
Subject: Re: [scalaz] Scala ByteString
I already have too many full-time jobs.  I'd definitely like to
squeeze every last drop of blood out of the JVM before going down that
path, but I don't want to rule anything out yet.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »