Seeking opinions on ScalaTest and Scalactic modularization

[Bill Venners]

Hi All,

At long last I can see light at the end of the ScalaTest/Scalactic 3.0 tunnel, which was primarily about adding Scala.js support. Scala.js support turned out to be a giant Trojan Yak that we've spent most of the last eight months shaving, but we're just about finished. It did force upon us several major enhancements that I either was never planning, or was hoping to do later, if at all. One of the outcomes was increased surface area: the 3.0 library has more stuff in it, primarily "async" and "safe" variants of all the style traits and classes.

I really struggled and fought to keep things as small as possible, trying out many different approaches, finally ending up with the basic organization of 3.0.0-M11. Scaladoc is here in case you want to hop around:

http://www.artima.com/docs-scalatest-3.0.0-M11/#org.scalatest.FlatSpec

Perhaps as an overreaction to me getting a feeling of adding too much new surface area, I decided to try out more modularization of the library. In the nine years of the ScalaTest project, I have only ever received two requests of modularization, so I don't think it is a searing user pain. Thus I wanted to ask for opinions before making the final decision about whether to do it.

One change we already made in 3.0 milestones is that the "scalatest" artifact that up to now everyone used in build files no longer contains a copy of Scalactic in the same jar. Instead it has a required transitive dependency to "org.scalactic"/"scalactic" artifact. Now there's a "scalatest-all" artifact that includes a copy of scalactic in it. The main purpose of this artifact is the quick start:

http://scalatest.org/quick_start

I.e., on that page I link to one jar someone can download to try out a bit of ScalaTest on the command line.

So essentially for anyone using "org.scalatest"/"scalatest" in their build, bumping the version to "3.0.0" is all that will be required, because now the build will follow the required dependency and download the appropriate Scalactic artifacts too.

The new modularization we are considering now is to have modules for dependencies to external libraries and to "choices." All modules will have group id "org.scalatest", but differ in artifact id. We include in ScalaTest APIs that interface with JUnit, TestNG, JMock, EasyMock, Mockito, Selenium. To use these, you could in 3.0.0 use "scalatest-all", "scalatest", or:

scalatest-junit

scalatest-testng

scalatest-jmock

scalatest-easymock

scalatest-mockito

scalatest-selenium

One thing we could do given this is a module is make the version dependency required instead of optional. I would like opinions on whether that's better or worse. In scalatest-all and scalatest, these dependencies would still be optional.

The "choices" modules are to enable projects to keep things consistent. There will be one module for each style plus separate modules for Matchers and MustMatchers. The style modules would be:

scalatest-featurespec
scalatest-flatspec
scalatest-freespec
scalatest-funspec
scalatest-funsuite
scalatest-propspec
scalatest-wordspec
scalatest-refspec

That last one for refspec is because I think we should give Spec a longer name and deprecate the old name. I'm not sure what to name it yet. Some ideas are RefSpec or ReSpec (for reflection-spec), DefSpec or MethodSpec (both referring to tests as methods). Any preferences? This style doesn't exist under ScalaTest.js, because there is no reflection on Scala.js, but it is the fastest one to compile so currently at least still useful on the JVM.

If you have decided to just use FunSuite with assertions in your project, then you would just make your dependency "scalatest-funsuite" and you're good to go. If you also use FeatureSpec in addition to FunSuite you just also include "scalatest-featurespec" as a dependency in your build. Then if someone tries to use FunSpec because they prefer that style personally, it won't compile unless they go in and add scalatest-funspec to the build. Likewise if they can't control their urge to write their favorite "should be" expressions, that wouldn't compile either unless they go into the build and add scalatest-matchers. The matcher modules would be:

scalatest-matchers // just gives you should, not must

scalatest-mustmatchers // just gives you must, not should

The point of the "choices" modules is to allow project to nudge their users to stay consistent with the testing style chosen for the project. I personally think such consistency makes code easier for teams to work on, and so that's the main benefit I think the choices modularization provides.

The only other benefit I can see of modularization is psychological. The jar file may seem rather big for a testing library. I don't think this really matters to anyone in practice, but humans being humans, I "feel" that some users might "feel" better about the surface area when they can select the smaller chunks of the library they actually plan to use. What do you think?

Along those lines, I am also considering whether to modularize scalactic offering these modules in addition to "scalactic", which has everything:

scalactic-equality

scalactic-or

scalactic-anyvals

One aspect of Scalactic that I think users like currently is how small and focused it is. We have added AnyVals in 3.0, and I foresee adding a few more features to it over the coming years, and again I may be overreacting, but I feel the tug of surface area increasing again. If someone just wants to use Or (and Every), but not the other stuff, then they they can just depend on "scalactic-or" in their project. Do you think this kind of "choice" modularization would be worthwhile in Scalactic?

Thanks.

Bill

----

Bill Venners
Artima, Inc.
http://www.artima.com

[Bill Venners]

Hi All,

I made a diagram showing the proposed modules and dependency graph and put it here:

http://artima.com/ScalaTestModularization.pdf

Bill

--

[Gavin Baumanis]

Hi Bill,

"I" like to treat my testing framework as a black-box that I can simply utilise, with the minimum of fuss.

I might never-ever use the selenium module, for example, but i will ALWAYS use the "ALL" jar, regardless.

It just makes it easy, in the sense that I have the whole black-box at my disposal - without having to think about which module I now needbecause my testing requirementts have changed.

And since you have on;y recieved two requests for making modules, I do think you could be a little "selfish",about thether ornot you bother.

That is; if it helps you to support / add features to Scalatest - then it would seem sensible to break it up.

If it is just a lot of extra work for an insignificant "plus" for you - "I" wouldn't bother.

Gavin.

[Bill Venners]

Hi Gavin,

Thanks for the feedback. One thing I hear is I should probably rename scalatest-all, because the artifact you probably want is just scalatest. The scalatest artifact has all of scalatest, and a required dependency on scalactic, which will be automatically followed by sbt, maven, ivy, etc. Not sure what to call scalatest-all, which *includes* a copy of scalactic in the same jar file, but the -all suffix is probably confusing.

The other thing is for us, it may actually make ScalaTest/Scalactic development easier because we might have more focused and thus likely quicker compile cycles. But I wouldn't have bothered for that purpose, at least not at this point.

Anyway, if we do go ahead with this, you can just continue to use the scalatest artifact in your build and you'll have everything like before.

Bill

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

[Gavin Baumanis]

Hi again,

I don’t think you need to worry about the naming.

I am sure it will be in the docs, right?

RTFM - when you need to know which modules you want - and you’re all good.

If separating it into modules allows for faster turnaround for changes - then that could be good.

Also - it might encourage others to help out in specific areas that suit them - and they won’t need to grok the “entire” library - just the single module they want to contribute to!

i still maintain - that I think “you” could be a little selfish about it - and do whatever is going to help you put the most.

As always 

Best regards,

Gavin Baumanis

gavinb...@gmail.com

You received this message because you are subscribed to a topic in the Google Groups "scalatest-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scalatest-users/s86u6Yn4CXI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scalatest-use...@googlegroups.com.

[Bill Venners]

Hi Gavin,

Thanks for saying that about being selfish. It is a good reminder, and moreover putting on your own oxygen mask first can mean you'll be better able to help others.

As far as asking users to read the manual, I do try to document thoroughly, but to the extent possible also try to make things so simple and obvious that it isn't necessary to read the manual. I just hit on a better name for scalatest-all, I think, which is scalatest-app. It's the artifact you use when you want to run ScalaTest as a standalone application. So "app". People might guess that's what scalatest-app means, but if not I think it is easy to remember once they look it up. And when looking at the artifacts, I think they'll guess scalatest is the one that gets them ScalaTest, and they'll be right. That's the artifact I'll show in the documentation, too, which has the other advantage that it will be the same as it has always been:

val scalatest = "org.scalatest" %% "scalatest" % scalatestVersion % "test"

That line says "scala" four times and "test" five times, so it almost screams that you're getting ScalaTest.

Bill