Account Options

  1. Sign in
Google Groups Home
« Groups Home
New ScalaTest 1.0-SNAPSHOT
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  9 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Bill Venners  
View profile  
 More options Sep 20 2009, 10:21 pm
From: Bill Venners <b...@artima.com>
Date: Sun, 20 Sep 2009 19:21:05 -0700
Local: Sun, Sep 20 2009 10:21 pm
Subject: New ScalaTest 1.0-SNAPSHOT
Hi All,

I just released a fresh ScalaTest 1.0 snapshot, both for 2.7.5 and the
latest nightly 2.8 build. Info on how to access it is here:

http://www.artima.com/scalatest/#snapshot

Some people have recently posted about trying to get ScalaTest 1.0 to
build under 2.8. It requires a few hand changes each time I do it,
because there is some code that simply won't compile under both 2.7
and 2.8. Each time I do this make a new branch, and from now on I will
check in the changes I make to get it to build under 2.8. If you want
to work with it, you can get tonight's build here:

svn checkout https://scalatest.dev.java.net/svn/scalatest/branches/app/trunk-for-s...
--username guest

I compiled it with the latest 2.8 nightly build (downloaded today):

scala-2.8.0.r18678-b20090910020815

The only tasks remaining are more testing of the
org.scalatest.concurrent package, a bit more work in the
org.scalatest.mock package, and documentation. With luck the next
snapshot will be a release candidate. If you have time please give it
a test drive and post feedback to scalatest-users.

Thanks.

Bill
----
Bill Venners
Artima, Inc.
http://www.artima.com


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Cay Horstmann  
View profile  
 More options Sep 21 2009, 12:33 am
From: Cay Horstmann <c...@horstmann.com>
Date: Sun, 20 Sep 2009 21:33:16 -0700 (PDT)
Local: Mon, Sep 21 2009 12:33 am
Subject: Re: New ScalaTest 1.0-SNAPSHOT
Thanks very much! It works like a charm for me. My students will be
delighted--I originally had them go back to 2.7.6, and that version's
Eclipse plugin seems to have more problems than the latest one.

ScalaTest has worked out very nicely for me for grading homework :-) I
really enjoyed the fact that I was able to choose the testing style
that was best for my situation.

Just one question--how do I suppress the exception trace when an
assertion fails? I mean, if I get a report that foo isn't actually
bar, that's good enough for me. I don't care to see the life story of
the test runner.

Thanks,

Cay

On Sep 20, 7:21 pm, Bill Venners <b...@artima.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Joakim Ohlrogge  
View profile  
 More options Sep 21 2009, 3:13 am
From: Joakim Ohlrogge <joakim.ohlro...@gmail.com>
Date: Mon, 21 Sep 2009 09:13:45 +0200
Local: Mon, Sep 21 2009 3:13 am
Subject: Re: [scalatest-users] New ScalaTest 1.0-SNAPSHOT

Tried it form eclipse and buildr. Works like charm (buildr reports that the
tests fail but that seems to be a problem with failure detection in
compination with using BDD-style (extends Spec)).
Thanks for staying up to date (eclipse only has the latest nightly build
available so new eclipse users will probably appreciate it :))

--
-----------------------------------------------------
Joakim Ohlrogge
Agical AB
Västerlånggatan 79, 2 tr
111 29 Stockholm, SWEDEN

Mobile: +46-708-754004
Blog: johlrogge.wordpress.com
E-mail: joakim.ohlro...@agical.se


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bill Venners  
View profile  
 More options Sep 21 2009, 9:01 am
From: Bill Venners <b...@artima.com>
Date: Mon, 21 Sep 2009 06:01:46 -0700
Local: Mon, Sep 21 2009 9:01 am
Subject: Re: [scalatest-users] Re: New ScalaTest 1.0-SNAPSHOT
Hi Cay,

On Sun, Sep 20, 2009 at 9:33 PM, Cay Horstmann <c...@horstmann.com> wrote:

> Thanks very much! It works like a charm for me. My students will be
> delighted--I originally had them go back to 2.7.6, and that version's
> Eclipse plugin seems to have more problems than the latest one.

> ScalaTest has worked out very nicely for me for grading homework :-) I
> really enjoyed the fact that I was able to choose the testing style
> that was best for my situation.

> Just one question--how do I suppress the exception trace when an
> assertion fails? I mean, if I get a report that foo isn't actually
> bar, that's good enough for me. I don't care to see the life story of
> the test runner.

Which reporter or reporters are you using? The only ones in which the
stack trace can really get in your face is the standard out, standard
err, or file reporters.

The story of stack traces is that I wanted to try and save people from
needing to scan through them to find the line of code for failed
tests, because that slows people down. When an assertion or matcher
expression fails, it does so with a TestFailedException, which mixes
in trait StackDepth. A StackDepth exception contains an integer stack
depth which is the index in the stack trace array of the line of code
of the failed assertion. There are other StackDepthExceptions as well,
JUnitTestFailedError for when you're using JUnit,
DuplicateTestNameException, etc. The reporters highlight the file name
and line number of the offending line of test code so you don't have
to scan, when the exception is a StackDepth. If it isn't a StackDepth,
such as if the code under test through some random exception, then the
reporters can only give you the stack trace.

However, I also show at least a truncated stack trace in the case of
StackDepths, because what can happen is you may factor out duplicate
code into a method. The method has an assertion that may fail, and if
you don't have any stack trace, you won't know which call site failed.
For example I have this helper method:

  def ensureTestFailedEventReceived(suite: Suite, testName: String) {
    val reporter = new EventRecordingReporter
    suite.run(None, reporter, new Stopper {}, Filter(), Map(), None,
new Tracker)
    val testFailedEvent =
reporter.eventsReceived.find(_.isInstanceOf[TestFailed])
    assert(testFailedEvent.isDefined)
    assert(testFailedEvent.get.asInstanceOf[TestFailed].testName === testName)
  }

If the second assertion fails, the stack depth will point to that
assertion. I'll also get the name of the test that failed, but I'll
still want to know more if I call this helper method multiple times
from the same test, like this:

test("something") {
  // ...
  ensureTestFailedEventReceived(spec1, "should blow up")
  // ...
  ensureTestFailedEventReceived(spec2, "should blow up")

}

Which one of these was the one that failed? spec1 or spec2? So that's
why I show around 7 lines of stack trace in the standard out, standard
err, and file reporters for StackDepth exceptions. I cut off the top,
which is above the stack depth, show the next seven lines of stack
trace, then truncate the rest. I'm figuring you'd have to do a lot of
factoring out into helper methods for those 7 lines of code to not be
enough.

If it isn't a StackDepth exception, you get the whole stack trace, and
if someone always want full stack traces, you can configure that by
adding a config param F to the reporter arg, like a -oF on the command
line will get you a standard out reporter that always shows full stack
traces for everything.

By the way, what RSpec, a Ruby BDD tool, does is put a footnote next
to a failure message, and then show the stack trace at the bottom.
When doing BDD like tests, a stack trace can make the output a bit
ugly. But I think when you have a failing test you really want to know
the info as quickly as possible, so I didn't want you to have to scan
for footnotes. By truncating it most of the time, my hope was that you
get quick access to the info without completely cluttering up your
nice spec-like output.

Bill

> Thanks,

> Cay

> On Sep 20, 7:21 pm, Bill Venners <b...@artima.com> wrote:
>> Hi All,

>> I just released a fresh ScalaTest 1.0 snapshot, both for 2.7.5 and the
>> latest nightly 2.8 build...If you have time please give it
>> a test drive and post feedback to scalatest-users.

--
Bill Venners
Artima, Inc.
http://www.artima.com

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bill Venners  
View profile  
 More options Sep 21 2009, 8:53 pm
From: Bill Venners <b...@artima.com>
Date: Mon, 21 Sep 2009 17:53:28 -0700
Local: Mon, Sep 21 2009 8:53 pm
Subject: Re: [scalatest-users] Re: New ScalaTest 1.0-SNAPSHOT
Hi Joakim,

A saw that you tweeted that with a patch to buildr you got this all
working under 2.8. Can you elaborate on what the patch was? I'd like
to understand what was needed.

Thanks.

Bill

On Mon, Sep 21, 2009 at 12:13 AM, Joakim Ohlrogge

--
Bill Venners
Artima, Inc.
http://www.artima.com

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Joakim Ohlrogge  
View profile  
 More options Sep 22 2009, 1:21 am
From: Joakim Ohlrogge <joakim.ohlro...@gmail.com>
Date: Tue, 22 Sep 2009 07:21:27 +0200
Local: Tues, Sep 22 2009 1:21 am
Subject: Re: [scalatest-users] Re: New ScalaTest 1.0-SNAPSHOT

Absolutely. Basically I did three things:
1. The obvious thing, change the version to be downloaded by buildr to
1.0-for-scala-2.8.0-SNAPSHOT
2. Change the maven group from org.scala-tools.testing to org.scalatest

These two changes were pretty straight forward though the group was not
parameterized in buildr while I could probably get away with changing the
scalatest-version by setting some variable in my project (but I didn't look
into that, i just hacked the scala/tests.rb file directly.

Now for the strangest thing I did: The buildr integration looked at
scalatests console output to determine if any tests failed. Since the output
looks a bit different when extending Spec I had to patch the condition that
checked if the run had competed.
3. I changed the regexp: /Run completed\./ to  /(Run completed\.)|(All tests
passed\.)/

I meant to ask you what would be the best way to do this since the buildr
integration I looked into (which is Daniel Spiewak's git-branch, unofficial
and probably work in progress) seems a bit hacky. Buildr uses the ant-task
under the hood, maybe I can get the same report output to file for all the
different runners and it would be a more reliable strategy to parse those?

Also the "completed" regexp seems to be there for this reason:

# Parse for failures, errors, etc.
# This is a bit of a pain right now because ScalaTest doesn't flush its
# output synchronously before the Ant test finishes so we have to loop
# and wait for an indication that the test run was completed.

So a flush at the end of a run (if 1.0 does not already do this) would maybe
speed things up a bit as well as relying only on the failure regexp:

/(TEST FAILED -)|(RUN STOPPED)|(RUN ABORTED)/

Which seems to work for the Spec style as well.

I meant to commit my changes to my git-branch so any suggestions for
improved strategies for failure detection etc would be most welcome.

You can have a look at my clone here:
http://github.com/johlrogge/buildr

At the moment I have not pushed any changes back so it is identical to
Daniel's clone.

Hope this helps
/J

--
-----------------------------------------------------
Joakim Ohlrogge
Agical AB
Västerlånggatan 79, 2 tr
111 29 Stockholm, SWEDEN

Mobile: +46-708-754004
Blog: johlrogge.wordpress.com
E-mail: joakim.ohlro...@agical.se


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bill Venners  
View profile  
 More options Sep 22 2009, 1:56 am
From: Bill Venners <b...@artima.com>
Date: Mon, 21 Sep 2009 22:56:56 -0700
Local: Tues, Sep 22 2009 1:56 am
Subject: Re: [scalatest-users] Re: New ScalaTest 1.0-SNAPSHOT
Hi Joakim,

There's actually a method in org.scalatest.tools.Runner intended for
this use case. Runner has a main method and a run method. The run
method takes the same args as main, but returns a Boolean. Here's its
doc comment:

Runs a suite of tests, with optional GUI. See the main documentation
for this singleton object for the details. The difference between this
method and main is simply that this method will block until the run
has completed, aborted, or been stopped, and return true if all tests
executed and passed. In other words, if any test fails, or if any
suite aborts, or if the run aborts or is stopped, this method will
return false. This value is used, for example, by the ScalaTest ant
task to determine whether to continue the build if haltOnFailure is
set to true.

Does it sound like buildr could call this method directly?

Bill

On Mon, Sep 21, 2009 at 10:21 PM, Joakim Ohlrogge

--
Bill Venners
Artima, Inc.
http://www.artima.com

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Joakim Ohlrogge  
View profile  
 More options Sep 22 2009, 3:27 am
From: Joakim Ohlrogge <joakim.ohlro...@gmail.com>
Date: Tue, 22 Sep 2009 09:27:59 +0200
Local: Tues, Sep 22 2009 3:27 am
Subject: Re: [scalatest-users] Re: New ScalaTest 1.0-SNAPSHOT

Thanks for the tip, I would have to look into that I'm not that comfortable
with the inner workings of buildr yet (or as cool people say: I'm a buildr
n00b :)). At the moment buildr uses the ant-task which from what I
understand is how things generally are done in buildr but it may be straight
forward. Or is there a flag for the ant-task that gives the same behavior?

/J

--
-----------------------------------------------------
Joakim Ohlrogge
Agical AB
Västerlånggatan 79, 2 tr
111 29 Stockholm, SWEDEN

Mobile: +46-708-754004
Blog: johlrogge.wordpress.com
E-mail: joakim.ohlro...@agical.se


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bill Venners  
View profile  
 More options Sep 22 2009, 3:56 am
From: Bill Venners <b...@artima.com>
Date: Tue, 22 Sep 2009 00:56:55 -0700
Local: Tues, Sep 22 2009 3:56 am
Subject: Re: [scalatest-users] Re: New ScalaTest 1.0-SNAPSHOT
Hi Joakim,

Yes, there's a haltonfailure flag in the ant task. That may be what
they'd rather use in buildr:

 <scalatest fork="true" runpath="${build.tests.jar}" parallel="true"
haltonfailure="true">

Bill

On Tue, Sep 22, 2009 at 12:27 AM, Joakim Ohlrogge

--
Bill Venners
Artima, Inc.
http://www.artima.com

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »