missing a good documentation

402 views
Skip to first unread message

François Gannaz

unread,
Jun 30, 2011, 5:43:05 AM6/30/11
to scala-user
Hi

As a newcomer, I'm often annoyed by Scala's documentation. I spend too much time
searching for info (StackOverflow, search engines). Sometimes, Scala's spartan
API doc is sufficient, but that's not often. I know I could always read the
source, but I'm too lazy!

I apologize for ranting. I know a documentation like Qt's isn't easy to build,
but I think it should be a priority for any language. I'm not qualified enough
to write definitive documentation, but I would be glad to contribute to a wiki.
When I see the many posts on SO and in blogs, I believe the community could
come up with a nice doc.

If you're interested in pratical situations that made me feel bad, here are a
few short cases. Feel free to blame me if I made mistakes.

I used some xml pattern matching, but soon discovered that it does not work with
attributes, so I dropped the whole lot. I got confirmation of this behaviour
from Daniel Spiewak's blog. It was not in the "authoritative reference" book,
nor in any user-friendly documentation I could find on the official site.

I use Jsoup to parse HTML. When I first tried to use a "map" on some iterator
returned by this java library, it failed miserably. I soon discovered that I
just had to add one line to fix this:
import scala.collection.JavaConversions._ // scala 2.8+
This info is not in the tutorials and manuals of scala-lang.org.

The official "Tour of Scala" still announces that "General RegExp patterns
[are] temporarily retracted from Scala". Anyway I needed regexps. The book was
not very detailed on this subject, so I had to rely on the API. It was painful.
Then I switched to Java's Pattern. I found it much easier to use. Probably
because it is well documented (I know very little Java).

A last example is parser combinators. They are indeed wonderful, but after a few
easy examples, I struggled with them until I could read by chance the excellent
PDF "Combinator Parsing in Scala" by Stephan Freund.

In a perfect world, I believe Scala's site would offer a general documentation,
in HTML, with a content similar to current books. The API would show for each
class a full description with perspective and comments. Each non-trivial method
would display at least some examples and "see also" links.

Regards
--
François

wookietreiber

unread,
Jun 30, 2011, 6:14:43 AM6/30/11
to scala...@googlegroups.com
I'd like to contribute to the scala documentation -- what would be the best way
to do so? (how to submit scala-doc patches -- what kind of access is required?)

best regards

> Fran�ois

Jan Goyvaerts

unread,
Jun 30, 2011, 6:19:24 AM6/30/11
to wookietreiber, scala...@googlegroups.com
That was one of the subjects in the ScalaTypes podcast #001 - http://scalatypes.com/interview-with-paul-phillips. I think.

On Thu, Jun 30, 2011 at 12:14, wookietreiber <kizkizz...@googlemail.com> wrote:
I'd like to contribute to the scala documentation -- what would be the best way
to do so? (how to submit scala-doc patches -- what kind of access is required?)

best regards
> François


Michael Klishin

unread,
Jun 30, 2011, 6:20:18 AM6/30/11
to François Gannaz, scala-user
François Gannaz escribió:

> In a perfect world, I believe Scala's site would offer a general documentation,
> in HTML, with a content similar to current books. The API would show for each
> class a full description with perspective and comments. Each non-trivial method
> would display at least some examples and "see also" links.François has a point here: Scala documentation isn't "bad", it just doesn't have guides beyond maybe a couple of
topics. But writing guides is the most time-consuming part of writing documentation, so it takes many years
for a (relatively) new language to read documentation guides quality level Python has, for example.

My 2¢.


MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin


Jan Goyvaerts

unread,
Jun 30, 2011, 6:40:59 AM6/30/11
to François Gannaz, scala-user
I share Franois' point : for newbies the standard API Scaladoc is frighting. (http://www.scala-lang.org/api/current/index.html) And I share also the point documentation is as important as code - as that's where the adoption of Scala starts - by newbies. :-)

Although I have to admit it's way better than the classic JavaDoc. You can search for certain classes and filter the extensive inheritage to your convenience. What I'm usually doing is to set the ordering to "by inherticance" rather than "alphabetic". It given a much cleaner overview where the class gets its mustard from. :-)

When I'm trying to find things out about the collections I'm often going to http://www.decodified.com/scala/collections-api.xml. It's probably me, but I think the schematic approach works a lot better.

One or two code examples would be very nice too. What's the typical use of the classes/types/traits/... ?

And I might be wrong, but I think it's a safe bet that a book about Scala Collections would sell well... :-)

2011/6/30 François Gannaz <francoi...@gmail.com>

Dave

unread,
Jun 30, 2011, 7:52:22 AM6/30/11
to scala-user
" When I'm trying to find things out about the collections I'm often
going to http://www.decodified.com/scala/collections-api.xml. It's
probably me, but I
think the schematic approach works a lot better."

That is indeed a good visual survey. A picture tells more than a 1000
words.
It's a known problem that Scala documentation is lagging and
fragmented.


Maybe on the arrows there could be selection criteria like:

- order preserving on addition or not
- every element is unique or duplicates are allowed
- the big O performance characteristic for
sequence type: head, tail, apply, update, prepend, append, insert
set and map typeslookup, add, remove, min
see: http://www.scala-lang.org/docu/files/collections-api/collections_40.html
(direct contents link for collections api updated 2.8 September 7,
2010: http://www.scala-lang.org/docu/files/collections-api/collections_0.html
)
- anyone using more criteria? I normally choose the immutable
collection over a mutable collection for the possibility of safe
parallellization.
Based on what criterium would anyone use a mutable collection over an
immutable collection? Probably the Big O in combination with what
volume of elements.
- Mutable/immutable is an obvious selection criterium already in the
picture

Big wish: A tool to select a collection based on criteria with weight
setting and then listing in order of preference? Maybe as part of
Scaladoc, the data for the criteria and extra selection info could be
annotated in the source file.

Randall R Schulz

unread,
Jun 30, 2011, 9:26:26 AM6/30/11
to scala...@googlegroups.com
On Thursday 30 June 2011, François Gannaz wrote:
> Hi
>
> ...

>
> A last example is parser combinators. They are indeed wonderful, but
> after a few easy examples, I struggled with them until I could read
> by chance the excellent PDF "Combinator Parsing in Scala" by Stephan
> Freund.

Any chance of getting an English translation of these slides?


> ...
> --
> François


Randall Schulz

François Gannaz

unread,
Jun 30, 2011, 9:57:05 AM6/30/11
to scala-user

Sorry, I mixed 2 references. I meant to write "Combinator Parsing". It is in
English, and it seems to be a chapter of a book (written at the EPFL?).

http://lamp.epfl.ch/teaching/foundations_of_software/docs/combinator_parsing.pdf

--
François

Randall R Schulz

unread,
Jun 30, 2011, 10:53:44 AM6/30/11
to scala...@googlegroups.com
On Thursday 30 June 2011, François Gannaz wrote:
> Hi
>
> ...
>
> The official "Tour of Scala" still announces that "General RegExp
> patterns [are] temporarily retracted from Scala". Anyway I needed
> regexps. The book was not very detailed on this subject, so I had to
> rely on the API. It was painful. Then I switched to Java's Pattern. I
> found it much easier to use. Probably because it is well documented
> (I know very little Java).

I'm sensing that you're confusing two different uses of regular
expressions. The ones that are in abeyance are for pattern matching in
the sense of Scala's "match" construct, not mere character string
processing. The latter exists with a small but helpful Scala class,
Regex, built over the Java standard library's java.util.regex classes.

Trond Olsen

unread,
Jun 30, 2011, 11:43:38 AM6/30/11
to Jan Goyvaerts, wookietreiber, scala...@googlegroups.com
Scala has podcasts? Thanks for the link.

Jan Goyvaerts

unread,
Jun 30, 2011, 12:16:43 PM6/30/11
to Trond Olsen, wookietreiber, scala...@googlegroups.com
In the last episode they're discussing the usefulness of better documentation btw.

Personally, I think there's progress to be made. Unless you don't want rookies to join of course. :-P

François Gannaz

unread,
Jun 30, 2011, 12:39:26 PM6/30/11
to scala-user
On 2011-06-30, Randall R Schulz <rsc...@sonic.net> wrote:

I've read again the page "A Tour of Scala: Regular Expression Patterns"
http://www.scala-lang.org/node/122

Now I think I understand what you mean. The "General RegExp patterns" are just
an extension of the "Right-ignoring sequence patterns" described in the first
paragraph. When I read it first, I thought they were totally different. The
first paragraph is about "patterns" on Seq[A] (obviously not a string regexp).
The second one writes about "regular expressions" on XML (looks very similar to
the usual string regexp, no ?). I think it is very easy to misunderstand this.

This is supposed to be a shiny page, showing the good sides of Scala to newbies
and curious programmers. I believe it is counter-productive as it is now. The
induced message "compatibility break with a feature removed" is another big
threat here.

About scala.util.Regex, I had a hard time to handle some simple tasks, like
getting a collection of the groups for all the matches of a pattern in a given
string. Something like ` push @a,$1 for /id=(\d+)/g ` in Perl.

--
François

Ken McDonald

unread,
Jun 30, 2011, 12:47:45 PM6/30/11
to scala...@googlegroups.com

The official "Tour of Scala" still announces that "General RegExp patterns
[are] temporarily retracted from Scala". Anyway I needed regexps. The book was
not very detailed on this subject, so I had to rely on the API. It was painful.
Then I switched to Java's Pattern. I found it much easier to use. Probably
because it is well documented (I know very little Java).


Those are not text-matching regexes. However, if you do a lot of regex work, could I modestly point you to http://kenmcdonald.github.com/rex ? It uses that underlying Scala/Java regex engine, but makes it much easier to reliably work with complex regexes.

Cheers,
Ken 

Francois

unread,
Jul 1, 2011, 8:03:25 AM7/1/11
to François Gannaz, scala-user
On 30/06/2011 11:43, François Gannaz wrote:
> Hi
>
> As a newcomer, I'm often annoyed by Scala's documentation. I spend too much time
> searching for info (StackOverflow, search engines). Sometimes, Scala's spartan
> API doc is sufficient, but that's not often. I know I could always read the
> source, but I'm too lazy!
> [...]

Not sure it has appeared in other thread, but there is at least two
books available online about Scala:

- Programming In Scala first edition : http://www.artima.com/pins1ed/
- Programming Scala : http://programming-scala.labs.oreilly.com/

They give a really good overall documentation on the langage.

Hope it helps,

--
Francois ARMAND
http://fanf42.blogspot.com
http://www.normation.com

john sullivan

unread,
Jul 1, 2011, 8:48:13 AM7/1/11
to François Gannaz, scala-user
About scala.util.Regex, I had a hard time to handle some simple tasks, like
getting a collection of the groups for all the matches of a pattern in a given
string. Something like ` push @a,$1 for /id=(\d+)/g ` in Perl.

Use positive lookbehind:

scala> val r = """(?<=id=)(\d+)""".r      
r: scala.util.matching.Regex = (?<=id=)(\d+)

scala> val m = r.findAllIn("xxid=77xxxid=45fdwgr")
m: scala.util.matching.Regex.MatchIterator = non-empty iterator

scala> m.foreach { println }                      
77
45

François Gannaz

unread,
Jul 1, 2011, 9:21:47 AM7/1/11
to scala-user

That's a nice trick, but it does not answer my needs.

I reckon it works on my simple Perl example, but it was a bit oversimplified.
As I wrote above, I want to capture the *groups*. Your trick is to make the
first group equal whole match. As soon as there are 2 groups, it fails.

Back to Perl: push @a,[$1,$2] while /id=(\d+):(\d+)/g

--
François

François Gannaz

unread,
Jul 1, 2011, 9:39:26 AM7/1/11
to scala-user

Thanks for this statement. I sounded too negative in my post. I should have
first written that Scala's official site has links to some very good
introductory books and manuals.

The problem is that as soon as one's need is not covered by these general books,
one has to rely on StackOverflow, blogs and the minimalistic documentation of
the API.

I know writing documentation is a tedious process. Many developers hate this.
So I believe scala-lang.org should have some sort of collaborative process
(wiki? dvcs?). It could begin by importing some known resources, if their
authors consent to this.

--
François Gannaz


Stefan Langer

unread,
Jul 1, 2011, 9:43:58 AM7/1/11
to François Gannaz, scala-user
Not sure exactly what you are trying to do as my perl is at best rusty ;)

but what about
val T = """id=(\d+):(\d+)""".r

input match {
case T(g1, g2) => println("group1 %s, group2 %s".format(g1,g2))
case _=> println("no match")
}


2011/7/1 François Gannaz <francoi...@gmail.com>:

Henry Ware

unread,
Jul 1, 2011, 9:58:31 AM7/1/11
to Stefan Langer, François Gannaz, scala-user
2011/7/1 Stefan Langer <mailto...@googlemail.com>:

> Not sure exactly what you are trying to do as my perl is at best rusty ;)
>
> but what about
> val T = """id=(\d+):(\d+)""".r
>
> input match {
>  case T(g1, g2) => println("group1 %s, group2 %s".format(g1,g2))
>  case _=> println("no match")
> }

Another way to use this regex is

scala> val s="""id=1:2
| id=3:4"""
s: java.lang.String =
id=1:2
id=3:4

scala> T.findAllIn(s).matchData.map(_.subgroups).toList
res3: List[List[String]] = List(List(1, 2), List(3, 4))

Hope that's helpful.

François Gannaz

unread,
Jul 1, 2011, 10:30:10 AM7/1/11
to scala-user

That's it! Thank you!

Had I know this was possible, I would not have bothered with Java's Pattern.

I first wrote a code like Stefan's, but this only finds the first match.
As I wanted to get all the groups for all the matches, I had to write a
tail-recursive function (heavy code for a simple need). Your solution is much
more elegant.

Thanks again, I'm now convinced to use Regex in the future
--
François Gannaz

Daniel Sobral

unread,
Jul 1, 2011, 10:34:01 AM7/1/11
to François Gannaz, scala-user
2011/6/30 François Gannaz <francoi...@gmail.com>:

> About scala.util.Regex, I had a hard time to handle some simple tasks, like
> getting a collection of the groups for all the matches of a pattern in a given
> string. Something like ` push @a,$1 for /id=(\d+)/g ` in Perl.

I missed this comment... Yes, I know *exactly* how you feel. Some ways
of doing it:

val a = for {
matched <- """id=(\d+)""".r.findAllIn(source).matchData.toList
} yield matched.subgroups(0)

val a = for {
matched <- """id=(\d+)""".r.findAllIn(source).matchData.toList
} yield matched.group(1)

val a = for {
matched <- """id=(\d+)""".r.findAllIn(source).matchData.toList
groups <- matched.subgroups
} yield groups

val a = """id=(\d+)""".r.findAllIn(source).matchData.flatMap(_ subgroups).toList

// The following examples need this import
import scala.util.matching.Regex.Groups

val a = for {
matched <- """id=(\d+)""".r.findAllIn(source).matchData.toList
Groups(first, _ @ _*) = matched
} yield first

val a = for {
Groups(first) <- """id=(\d+)""".r.findAllIn(source).matchData.toList
} yield first

// If you have many subgroups, but only care about the first
val a = for {
Groups(first, _ @ _*) <- """id=(\d+)""".r.findAllIn(source).matchData.toList
} yield first

// The following example needs the pattern to be assigned to an identifier
val pattern = """id=(\d+)""".r
val a = for {
pattern(first) <- pattern.findAllIn(source).matchData.toList
} yield first


--
Daniel C. Sobral

I travel to the future all the time.

Florian Hars

unread,
Jul 1, 2011, 10:53:19 AM7/1/11
to François Gannaz, y...@st11.hars.de, scala-user
On Fri, Jul 01, 2011 at 03:21:47PM +0200, François Gannaz wrote:

> As I wrote above, I want to capture the *groups*.

Well, then use the group method on the matches.

> Back to Perl: push @a,[$1,$2] while /id=(\d+):(\d+)/g

scala> """id=(\d+):(\d+)""".r.findAllIn("xxid=77:33xxxid=45:44fdwgr").matchData.toList map (md => (md.group(1), md.group(2)))
res4: List[(String, String)] = List((77,33), (45,44))

Yeah, perl has shorter oneliners.

- Florian.
--
#!/bin/sh -
set - `type -p $0` 'tr [a-m][n-z]RUXJAKBOZ [n-z][a-m]EH$W/@OBM' fu XUBZRA.fvt\
angher echo;while [ "$5" != "" ];do shift;done;$4 "gbhpu $3;fraqznvy sKunef.q\
r<$3&&frq -a -rc "`$4 "$0"|$1`">$3;rpub 'Jr ner Svt bs Obet.'"|$1|`$4 $2|$1`

Daniel Sobral

unread,
Jul 1, 2011, 11:02:01 AM7/1/11
to François Gannaz, scala-user
2011/7/1 François Gannaz <francoi...@gmail.com>:

>
> I know writing documentation is a tedious process. Many developers hate this.
> So I believe scala-lang.org should have some sort of collaborative process
> (wiki? dvcs?). It could begin by importing some known resources, if their
> authors consent to this.

As it happens, it has a wiki. The old one had a wiki too, with some
esoteric stuff mixed in with some basic stuff, but not in any way
comprehensive. And, then, there's colladoc, which is a way to submit
changes to Scaladoc itself. The tool was never integrated in any way
for these changes to be reliably accepted, however. And, then,t here's
a *fantastic* on-line resource for learning Scala, which is Simply
Scala, because it let you execute and *interact* with all the code it
presents.

Old wiki: http://scala.sygneca.com/
New wiki: https://wiki.scala-lang.org/
Colladoc: http://scala-webapps.epfl.ch/colladoc/
Colladoc project: http://collaborative-scaladoc.github.com/
Simply Scala: http://www.simplyscala.com/

So, here's the deal... there has been a wiki for many years, but it
hasn't been used or updated. There's colladoc to help with API docs
itself too. The question is: why this hasn't been enough?

Personally, I can think of a few things. The wiki, for one thing, is
not on scala-lang or epfl.ch. But, more importantly, it is not *the*
scala-lang! When you look at succesful wiki projects in other
languages, the wiki is not some link hidden somewhere. On PHP.Net, the
link to documentation is topmost, and the documentation *is* the wiki.
It is not the pattern everyone takes -- Ruby is also a huge collection
of links (mind you, Ruby is an old language that took a long time to
become popular *because its documentation sucked*).

But Scala... it's bureaucratic. Documentation is not a link (well, it
is too, but doesn't act like one), it is a menu under which there are
items and submenus. The items taken you to pages with links to other
places. And, then, there's the manuals, which are all PDF-only (and
not particularly searchable or linkable to because of that).

And, for all that, I, for one, didn't see a link to either wiki, to
colladoc, or to simply scala.

Honestly, this all gets me even more depressed. There's so much that's
within easy reach, some stuff that could incredibly exciting -- it is
a simple lack of *action*.

I don't blame the EPFL people. The task of turning scala-lang into a
site where documention is easy to find, to contribute to, and good in
quality is not a research task. Colladoc is research, putting it to
use is not.

*This* is the difference that a company backing makes. This is why
Java has good documentation. This is why I expect Ceylon to have good
documentation. I hope Typesafe can do something about this, but, to be
completely honest, I'm doubtful.

Reply all
Reply to author
Forward
0 new messages