'doc' command in sbt 0.12

[Mark Harrah]

There is what I think is a blocking issue with the new behavior of the `doc` command in sbt 0.12 that needs to be resolved in an RC4. `doc` in 0.12 runs javadoc on Java sources and scaladoc on Java and Scala sources. When there are only Java sources or only Scala sources but not both, the output directory is .../api/ as before. When there are both Java and Scala sources present, the output directory has to be more specific: api/java for Java and api/scala for Scala because both have `index.html`. The result of `doc` is `api/` in all cases, however. This was done to preserve the `File` return type of the `doc` task, but I think this needs to be handled better.

1. Mixed doc generation should be better configurable. Currently, the behavior is autodetected based on sources. That can be the default still, but it should be possible to enable or disable javadoc and/or scaladoc explicitly.

2. The output of `doc` should be a richer structure that at a minimum, provides the Javadoc output directory and Scaladoc output directory.

Opinions are welcome.

-Mark

[Seth Tisue]

On Mon, Jul 16, 2012 at 12:07 PM, Mark Harrah <dmha...@gmail.com> wrote:
> `doc` in 0.12 runs javadoc on Java sources and scaladoc on Java and Scala sources.

For my project, I want to generate Scaladoc for all sources, both
Scala and Java.

I'm not sure if it's widely known that Scaladoc can do that. But it
can! It doesn't actually process either Javadoc or Scaladoc comments
in Java source code, but at least it generates skeletal Scaladoc that
has the right method names and types.

Since that isn't how `doc` behaves in sbt, so I copy-and-pasted the
source code for the task and altered it to do what I want. Gory,
don't-try-this-at-home details at
https://github.com/NetLogo/NetLogo/blob/master/build.sbt#L62-110

Anyway, what I'm trying to say is that it would welcome if `doc` was
more configurable, and obviously it would be especially welcome if my
particular use case were supported :-) Shall I open a ticket? (And,
oh no, my bad for not being aware of
https://github.com/harrah/xsbt/issues/281 until now.)

(Is any of this really blocking for 0.12, though? I'm eager to see
0.12 go final.)

--
Seth Tisue | Northwestern University | http://tisue.net
lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/

[Seth Tisue]

On Mon, Jul 16, 2012 at 12:21 PM, Seth Tisue <se...@tisue.net> wrote:
> For my project, I want to generate Scaladoc for all sources, both
> Scala and Java.

Oh, I guess I should explain why this can be desirable.

I consider my codebase primarily a Scala codebase. Most Java stuff in
it is still Java only for historical reasons (haven't gotten around to
converting it). But there are other reasons, too. Some files are in
Java because it's third party code it seems better not to touch, some
files are in Java because they're enums and I don't like Scala enums,
some files need maximum performance which depending on details is
sometimes easier to get with Java... and so on.

But because it's still primarily a Scala codebase, I want to have a
single, unified Scaladoc to browse, not two separate docs where you
have to know an accidental fact about a class (whether it has been
converted to Scala yet) to know where to find it.

[Mark Harrah]

On Mon, 16 Jul 2012 12:21:26 -0400
Seth Tisue <se...@tisue.net> wrote:

> On Mon, Jul 16, 2012 at 12:07 PM, Mark Harrah <dmha...@gmail.com> wrote:
> > `doc` in 0.12 runs javadoc on Java sources and scaladoc on Java and Scala sources.
>
> For my project, I want to generate Scaladoc for all sources, both
> Scala and Java.
>
> I'm not sure if it's widely known that Scaladoc can do that. But it
> can! It doesn't actually process either Javadoc or Scaladoc comments
> in Java source code, but at least it generates skeletal Scaladoc that
> has the right method names and types.
>
> Since that isn't how `doc` behaves in sbt, so I copy-and-pasted the
> source code for the task and altered it to do what I want. Gory,
> don't-try-this-at-home details at
> https://github.com/NetLogo/NetLogo/blob/master/build.sbt#L62-110

'doc' should pass your Java sources to scaladoc as well as to javadoc by default. It will put the scaladoc output under api/scala/ and the javadoc output under api/java/.

> Anyway, what I'm trying to say is that it would welcome if `doc` was
> more configurable, and obviously it would be especially welcome if my
> particular use case were supported :-) Shall I open a ticket? (And,
> oh no, my bad for not being aware of
> https://github.com/harrah/xsbt/issues/281 until now.)
>
> (Is any of this really blocking for 0.12, though? I'm eager to see
> 0.12 go final.)

I don't want an RC4 either, but this is the time to fix it. There hasn't been a release with this change, this change can be surprising, and there isn't an easy switch to modify the behavior (I think your copy-and-paste shows this is a real problem!). Once it is in 0.12.0 it probably can't be addressed until 0.13.0 for compatibility reasons.

-Mark

> --
> Seth Tisue | Northwestern University | http://tisue.net
> lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
>

> --
> You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
> To post to this group, send email to simple-b...@googlegroups.com.
> To unsubscribe from this group, send email to simple-build-t...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en.
>

[Mark Harrah]

On Mon, 16 Jul 2012 12:26:27 -0400
Seth Tisue <se...@tisue.net> wrote:

> On Mon, Jul 16, 2012 at 12:21 PM, Seth Tisue <se...@tisue.net> wrote:
> > For my project, I want to generate Scaladoc for all sources, both
> > Scala and Java.
>
> Oh, I guess I should explain why this can be desirable.

I do this for sbt as well despite, as you say, the Javadoc for Java sources not being processed. For a mixed Java/Scala project, the code in RC3 will always pass Java sources to scaladoc. This would most likely be the default in any revised version as well.

-Mark

> I consider my codebase primarily a Scala codebase. Most Java stuff in
> it is still Java only for historical reasons (haven't gotten around to
> converting it). But there are other reasons, too. Some files are in
> Java because it's third party code it seems better not to touch, some
> files are in Java because they're enums and I don't like Scala enums,
> some files need maximum performance which depending on details is
> sometimes easier to get with Java... and so on.
>
> But because it's still primarily a Scala codebase, I want to have a
> single, unified Scaladoc to browse, not two separate docs where you
> have to know an accidental fact about a class (whether it has been
> converted to Scala yet) to know where to find it.
>
> --
> Seth Tisue | Northwestern University | http://tisue.net
> lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
>

[Josh Suereth]

What if we just generate only Scaladoc with java source for now and delay again until 0.13 :)

Barring that, I think we need to add configurable settings for what to run and what directories are made.   I need to be aware of this when generating project websites.

[Seth Tisue]

On Mon, Jul 16, 2012 at 12:42 PM, Mark Harrah <dmha...@gmail.com> wrote:
> I do this for sbt as well despite, as you say, the Javadoc for Java sources not being processed. For a mixed Java/Scala project, the code in RC3 will always pass Java sources to scaladoc. This would most likely be the default in any revised version as well.

OK. It's probably already clear, but: what I want is for the output
for both input languages to come out together in api/, not separated
into api/scala and api/java.

[Mark Harrah]

On Mon, 16 Jul 2012 14:57:23 -0400
Seth Tisue <se...@tisue.net> wrote:

> On Mon, Jul 16, 2012 at 12:42 PM, Mark Harrah <dmha...@gmail.com> wrote:
> > I do this for sbt as well despite, as you say, the Javadoc for Java sources not being processed. For a mixed Java/Scala project, the code in RC3 will always pass Java sources to scaladoc. This would most likely be the default in any revised version as well.
>
> OK. It's probably already clear, but: what I want is for the output
> for both input languages to come out together in api/, not separated
> into api/scala and api/java.

Yes, this is how it was in 0.11 and earlier. We can't put both javadoc and scaladoc output in api/ because they both generate index.html, for example. You'd only be able to run scaladoc (which is what you want and until Scaladoc can link to external documentation, it is what I want) or javadoc would have to go somewhere else.

These two options are what have been proposed so far: Josh suggests keeping the current behavior, which only runs scaladoc and puts it in api/. I suggest at least making it configurable, so scaladoc could go in api/ and javadoc in java-api/ or something. A third option is to always put javadoc in java-api/.

-Mark

> --
> Seth Tisue | Northwestern University | http://tisue.net
> lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
>

[Seth Tisue]

The RC4 changes are working well for me with NetLogo. I'm now able to use the standard task, as is, to generate my full API doc. Yay!

(I do still have a custom doc task that runs Scaladoc a second time with a smaller set of sources and a different output directory, but that's another story.)

[Mark Harrah]

On Wed, 18 Jul 2012 15:49:13 -0700 (PDT)
Seth Tisue <se...@tisue.net> wrote:

> The RC4 changes are working well for me with NetLogo. I'm now able to use
> the standard task, as is, to generate my full API doc. Yay!

Good, thanks for the feedback.

> (I do still have a custom doc task that runs Scaladoc a second time with a
> smaller set of sources and a different output directory, but that's another
> story.)

You should be able to configure this without copying/pasting the 'doc' implementation (in case you are doing so and would prefer not to).

-Mark

> --
> Seth Tisue | Northwestern University | http://tisue.net
> lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
>

> --
> You received this message because you are subscribed to the Google Groups "simple-build-tool" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/simple-build-tool/-/nGDU52HTEjgJ.

[Seth Tisue]

On Fri, Jul 20, 2012 at 1:04 PM, Mark Harrah <dmha...@gmail.com> wrote:
>> (I do still have a custom doc task that runs Scaladoc a second time with a
>> smaller set of sources and a different output directory, but that's another
>> story.)
>
> You should be able to configure this without copying/pasting the 'doc' implementation (in case you are doing so and would prefer not to).

I am doing so, and it isn't too awful. So the following just comes
from my own desire to experiment and learn, rather than from any
actual pressing need.

I hit a roadblock while trying to following your suggestions at
Scalathon on avoiding the copy/paste.

As I said, in my custom doc task I'm passing only a subset of the
sources. But those sources refer to classes that *aren't* defined in
the subset. So when I call Scaladoc, I need to make sure that the
classpath includes the compiled versions of those classes, otherwise
doc generation fails with a million class not found errors.

But in Defaults.scala, in the body of the docTaskSettings method, we
see the following:

val cp = in.config.classpath.toList - in.config.classesDirectory

So it's removing exactly the thing I need included.

I don't see a way around this without patching sbt itself. Do you agree?

Also, I was getting this error:

[error] References to undefined settings:
[error] *:doc-smaller::configuration from *:doc-smaller

and I managed to get it to go away by adding:

configuration in docSmaller := Configurations.Compile

but I'm wondering if that's actually correct.

[Mark Harrah]

On Thu, 2 Aug 2012 13:23:01 -0400
Seth Tisue <se...@tisue.net> wrote:

> On Fri, Jul 20, 2012 at 1:04 PM, Mark Harrah <dmha...@gmail.com> wrote:
> >> (I do still have a custom doc task that runs Scaladoc a second time with a
> >> smaller set of sources and a different output directory, but that's another
> >> story.)
> >
> > You should be able to configure this without copying/pasting the 'doc' implementation (in case you are doing so and would prefer not to).
>
> I am doing so, and it isn't too awful. So the following just comes
> from my own desire to experiment and learn, rather than from any
> actual pressing need.
>
> I hit a roadblock while trying to following your suggestions at
> Scalathon on avoiding the copy/paste.
>
> As I said, in my custom doc task I'm passing only a subset of the
> sources. But those sources refer to classes that *aren't* defined in
> the subset. So when I call Scaladoc, I need to make sure that the
> classpath includes the compiled versions of those classes, otherwise
> doc generation fails with a million class not found errors.
>
> But in Defaults.scala, in the body of the docTaskSettings method, we
> see the following:
>
> val cp = in.config.classpath.toList - in.config.classesDirectory
>
> So it's removing exactly the thing I need included.
>
> I don't see a way around this without patching sbt itself. Do you agree?

You could probably redefine classDirectory, but it is obviously a hack and the benefit might not be worth it. So yes, I essentially agree. Sorry for the diversion.

-Mark

> Also, I was getting this error:
>
> [error] References to undefined settings:
> [error] *:doc-smaller::configuration from *:doc-smaller
>
> and I managed to get it to go away by adding:
>
> configuration in docSmaller := Configurations.Compile
>
> but I'm wondering if that's actually correct.
>
> --
> Seth Tisue | Northwestern University | http://tisue.net
> lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
>

> --
> You received this message because you are subscribed to the Google Groups "simple-build-tool" group.

[Seth Tisue]

On Thu, Aug 2, 2012 at 3:25 PM, Mark Harrah <dmha...@gmail.com> wrote:
> You could probably redefine classDirectory, but it is obviously a hack and the benefit might not be worth it. So yes, I essentially agree. Sorry for the diversion.

np, it was diverting in the good sense.