I want to announce managed fixture -- reimplementation of scalatest
fixture tests to situation, where our fixture have external state
(like relational database) and test author can
- mark each test with description of state preconditions and changes
in simple DSL
- provide functions for loading given state.
Then test engine can split all tests into parts, which require same
state and can be executed in parallel, then execute this parts in
order of execution, which minimize state loading. I.e. tests can be
written in next style:
<pre>
class MyFlatSpec extends managedfixture.[DBFixtureStateTypes.type]
{
val fuxtureStateTypes = DBFixtureStateTypes
val fuxtureAccess = DBFixtureAccess
import DBFixtureStateTypes.States._
behavior of "My datababase"
start state(INITIAL) finish state(WITH_ONE)
it should "be able to add user " in { db =>
.....
}
start state(DATASET1) change(nothing) parallel
it should "retrieve user with name Jon in our test dataset" in { db
=>
inTransaction {
val x = db.selectUser("Jon").headOption;
assert(x!=None)
assert(x.name == "Jon")
}
}
</pre>
It's now in experimental stage: works at first glance but not yet
fully tested.
In future I want to submit all into one of next version of scalatest,
if this is possible.
So, question - if this possible (?), and if yes -- how to do this
(?)
[May be mirror on github for pull requests ?]
Also it would be good to see some changes in next scalatest core:
- now managed-fixture suites contains reimplementations of each
DSL. This can be simplicifed if scalatest core would allows hook
arround registerTest .. registerBranch calls in fixture suite.
- distruptors - we can't load different database states at the
same time from different tests, so we need from distruptors ability to
run set of tasks inside external lock or may-be just callback call
when all tests in distruptors are finished [?]
// As alternative, we can implement own parallel execution scheme,
does not depend from distruptors.
That looks interesting. From what I gather, you want to "annotate" tests with a bit of code (in a DSL) that defines what fixture stuff should happen before and after a test. Is that an accurate summary?
I'm guessing that "start state (<value>)" means that before the test starts, the passed state should be setup. I'm not sure what "finish state(WITH_ONE)" means. Can you clarify? Also, what does "change(nothing)" mean? And what about "parallel"?
With "parallel" do you want to mark those tests that can run in parallel as opposed to those that can't perhaps?
As far as github. I have considered having a mirror over there of major releases. Almost like a tag, that people could do pull requests against. I'll check into how easy that is. The reason I didn't move over to begin with when I moved to Google code from java.net is I didn't want to lose all the history. But it would be nice if people could fork a release on GitHub and submit a pull request. Especially if there was some easy way for me to get that back into Google code.
As far as Distributors go, I believe you can define your own distributor that has the features you need. You'd need to downcast it with a pattern match to your more specific type, but I think that would be the way to do it. Why don't you try implementing that so I can see what it looks like. It is not impossible that we could add such a thing to ScalaTest, but I would want to make sure it is very very well understood before it goes in. I need something like that in the afterAll method, for example.
I'm not too keen on opening up registerTest and registerBranch. Those are not generic enough to be lifecycle methods. For one thing, they don't work on traits where tests are methods like Suite (and in the future, the new Spec). For another example look at the new path traits. Those methods don't work on them either. Instead what you've done is actually the intended way to customize: you made some custom style traits. They look like the built-in ones but support your managed fixture feature.
That said, the other thing I was wondering about is did you consider just executing the DSL code inside the test body? Instead of:
start state(INITIAL) finish state(WITH_ONE) it should "be able to add user " in { db => //...
}
It might be something like:
it should "be able to add user " in { withFixture(start state(INITIAL) finish state(WITH_ONE)) { db => // ... }
}
This withFixture method you could place in a trait and users could mix that into any style trait, even path traits or traits with test methods, JUnit, TestNG, and so on. A lot more general. This probably doesn't cover the parallel thing, but that depends on what you were trying to achieve there. But I think it could take care of the setup and teardown stuff if I'm understanding correctly what you're trying to do.
On Sun, Feb 26, 2012 at 11:09 PM, rssh <ruslan.s.shevche...@gmail.com> wrote:
> Good day, colleagues.
> I want to announce managed fixture -- reimplementation of scalatest > fixture tests to situation, where our fixture have external state > (like relational database) and test author can > - mark each test with description of state preconditions and changes > in simple DSL > - provide functions for loading given state.
> Then test engine can split all tests into parts, which require same > state and can be executed in parallel, then execute this parts in > order of execution, which minimize state loading. I.e. tests can be > written in next style:
> <pre> > class MyFlatSpec extends managedfixture.[DBFixtureStateTypes.type] > { > val fuxtureStateTypes = DBFixtureStateTypes > val fuxtureAccess = DBFixtureAccess > import DBFixtureStateTypes.States._
> behavior of "My datababase"
> start state(INITIAL) finish state(WITH_ONE) > it should "be able to add user " in { db => > ..... > }
> start state(DATASET1) change(nothing) parallel > it should "retrieve user with name Jon in our test dataset" in { db > => > inTransaction { > val x = db.selectUser("Jon").headOption; > assert(x!=None) > assert(x.name == "Jon") > } > } > </pre>
> It's now in experimental stage: works at first glance but not yet > fully tested.
> In future I want to submit all into one of next version of scalatest, > if this is possible. > So, question - if this possible (?), and if yes -- how to do this > (?) > [May be mirror on github for pull requests ?]
> Also it would be good to see some changes in next scalatest core:
> - now managed-fixture suites contains reimplementations of each > DSL. This can be simplicifed if scalatest core would allows hook > arround registerTest .. registerBranch calls in fixture suite.
> - distruptors - we can't load different database states at the > same time from different tests, so we need from distruptors ability to > run set of tasks inside external lock or may-be just callback call > when all tests in distruptors are finished [?] > // As alternative, we can implement own parallel execution scheme, > does not depend from distruptors.
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
On Monday, 27 February 2012 22:34:20 UTC+2, Bill Venners wrote:
> As far as github. I have considered having a mirror over there of > major releases. Almost like a tag, that people could do pull requests > against. I'll check into how easy that is. The reason I didn't move > over to begin with when I moved to Google code from java.net is I > didn't want to lose all the history. But it would be nice if people > could fork a release on GitHub and submit a pull request. Especially > if there was some easy way for me to get that back into Google code.
Actually the problem was the *size* of the history. ScalaTest started as a "port" from the Java test framework SuiteRunner, which was started in 2001, thus it still had jar files checked in. So ScalaTest's history includes a lot of jar files, and that is impractical import into git. I did try it before deciding to move to Google Code, and it never completed the import. So either I had to let go of the history and start fresh, or stick with SVN. I chose the latter. But I think I'd like to copy release tags over to my github account so people can more easily contribute that way.
Say, did you get a chance to look at the path traits? Any suggestions for improvements?
On Tue, Feb 28, 2012 at 1:32 PM, Esko Luontola <esko.luont...@gmail.com> wrote: > On Monday, 27 February 2012 22:34:20 UTC+2, Bill Venners wrote:
>> As far as github. I have considered having a mirror over there of >> major releases. Almost like a tag, that people could do pull requests >> against. I'll check into how easy that is. The reason I didn't move >> over to begin with when I moved to Google code from java.net is I >> didn't want to lose all the history. But it would be nice if people >> could fork a release on GitHub and submit a pull request. Especially >> if there was some easy way for me to get that back into Google code.
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
I've also looked into this. The main problem with Scalatest is that the SVN branch structure is very much non-standard, so it's not particularly amenable to any automated conversion.
On 28 February 2012 21:38, Bill Venners <b...@artima.com> wrote:
> Actually the problem was the *size* of the history. ScalaTest started > as a "port" from the Java test framework SuiteRunner, which was > started in 2001, thus it still had jar files checked in. So > ScalaTest's history includes a lot of jar files, and that is > impractical import into git. I did try it before deciding to move to > Google Code, and it never completed the import. So either I had to let > go of the history and start fresh, or stick with SVN. I chose the > latter. But I think I'd like to copy release tags over to my github > account so people can more easily contribute that way.
> Say, did you get a chance to look at the path traits? Any suggestions > for improvements?
> Thanks.
> Bill
> On Tue, Feb 28, 2012 at 1:32 PM, Esko Luontola <esko.luont...@gmail.com> > wrote: > > On Monday, 27 February 2012 22:34:20 UTC+2, Bill Venners wrote:
> >> As far as github. I have considered having a mirror over there of > >> major releases. Almost like a tag, that people could do pull requests > >> against. I'll check into how easy that is. The reason I didn't move > >> over to begin with when I moved to Google code from java.net is I > >> didn't want to lose all the history. But it would be nice if people > >> could fork a release on GitHub and submit a pull request. Especially > >> if there was some easy way for me to get that back into Google code.
> > -- > > You received this message because you are subscribed to the Google > > Groups "scalatest-users" group. > > To post to this group, send email to scalatest-users@googlegroups.com > > To unsubscribe from this group, send email to > > scalatest-users+unsubscribe@googlegroups.com > > For more options, visit this group at > > http://groups.google.com/group/scalatest-users?hl=en > > ScalaTest itself, and documentation, is available here: > > http://www.artima.com/scalatest
I've been wanting to clean out all the dead branches. I haven't had time, but most are unused. For the past year or so, we have been putting 2.0 changes into the trunk, and doing 1.7 and 1.8 changes in their own branches, but once 1.8 goes final thankfully we'll be back in the trunk trunk heading to 2.0. Other than that, we tag release as they come out under /tags, and that's pretty standard I think for svn. These are the dirs I was thinking we'd pull into github, each tagged release. Any other suggestions other than removing old branches once we're done with them?
On Tue, Feb 28, 2012 at 2:08 PM, Kevin Wright <kev.lee.wri...@gmail.com> wrote: > I've also looked into this. The main problem with Scalatest is that the SVN > branch structure is very much non-standard, so it's not particularly > amenable to any automated conversion.
> On 28 February 2012 21:38, Bill Venners <b...@artima.com> wrote:
>> Hi Esko,
>> Actually the problem was the *size* of the history. ScalaTest started >> as a "port" from the Java test framework SuiteRunner, which was >> started in 2001, thus it still had jar files checked in. So >> ScalaTest's history includes a lot of jar files, and that is >> impractical import into git. I did try it before deciding to move to >> Google Code, and it never completed the import. So either I had to let >> go of the history and start fresh, or stick with SVN. I chose the >> latter. But I think I'd like to copy release tags over to my github >> account so people can more easily contribute that way.
>> Say, did you get a chance to look at the path traits? Any suggestions >> for improvements?
>> Thanks.
>> Bill
>> On Tue, Feb 28, 2012 at 1:32 PM, Esko Luontola <esko.luont...@gmail.com> >> wrote: >> > On Monday, 27 February 2012 22:34:20 UTC+2, Bill Venners wrote:
>> >> As far as github. I have considered having a mirror over there of >> >> major releases. Almost like a tag, that people could do pull requests >> >> against. I'll check into how easy that is. The reason I didn't move >> >> over to begin with when I moved to Google code from java.net is I >> >> didn't want to lose all the history. But it would be nice if people >> >> could fork a release on GitHub and submit a pull request. Especially >> >> if there was some easy way for me to get that back into Google code.
>> > It's quite easy to convert an SVN repository to Git and keep all the
>> > -- >> > You received this message because you are subscribed to the Google >> > Groups "scalatest-users" group. >> > To post to this group, send email to scalatest-users@googlegroups.com >> > To unsubscribe from this group, send email to >> > scalatest-users+unsubscribe@googlegroups.com >> > For more options, visit this group at >> > http://groups.google.com/group/scalatest-users?hl=en >> > ScalaTest itself, and documentation, is available here: >> > http://www.artima.com/scalatest
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
On Tuesday, 28 February 2012 23:38:20 UTC+2, Bill Venners wrote:
> Actually the problem was the *size* of the history. ScalaTest started > as a "port" from the Java test framework SuiteRunner, which was > started in 2001, thus it still had jar files checked in. So > ScalaTest's history includes a lot of jar files, and that is > impractical import into git. I did try it before deciding to move to > Google Code, and it never completed the import.
Did you try importing on your own machine, or using GitHub's import service? The former should be more reliable. Even more reliable is if you run the SVN repository on localhost, to avoid network problems and make it a bit faster.
After importing, it's possible to filter the history to remove the JAR files.
Say, did you get a chance to look at the path traits? Any suggestions
> for improvements?
I read the documentation. Looks good.
The documentation about ignored tests says "Note that a separate instance will be created for an ignored test". Does this mean that the fixture setup will be executed, but then no tests are executed? If the ignored test is followed by siblings, then it should be possible to execute one of them, to avoid wasting the effort which was spent on running the fixture setup.
On Wednesday, 29 February 2012 00:08:15 UTC+2, Kevin Wright wrote:
> I've also looked into this. The main problem with Scalatest is that the > SVN branch structure is very much non-standard, so it's not particularly > amenable to any automated conversion.
Git is very flexible in that regard (though it would require more or less manual work). The history can be modified to fix any weirdness, and even merges can be added to histories from pre-1.5 SVN repositories using "Git grafts".
I can have a look at the branch structure and estimate how much effort would be needed to import it to Git and make it clean.
On Wednesday, 29 February 2012 00:50:17 UTC+2, Esko Luontola wrote:
> On Wednesday, 29 February 2012 00:08:15 UTC+2, Kevin Wright wrote:
>> I've also looked into this. The main problem with Scalatest is that the >> SVN branch structure is very much non-standard, so it's not particularly >> amenable to any automated conversion.
> I can have a look at the branch structure and estimate how much effort > would be needed to import it to Git and make it clean.
I had a quick look at the repository. Appears that during java.net the source was under /trunk/app and now the source is under /trunk. The branches have some non-standard structures, for example I see at least these styles: /branches/*/app/, /branches/app/*/, /branches/*/ and also some weird mistakes such as /branches/app/bv-1/app/ (branch inside another branch).
It would be possible to clean up the history so that /app is removed from the beginning (with git filter-branch), the branches are imported correctly (with git filter-branch), and merges are added to the pre-1.5 SVN "merges" (with git grafts and filter-branch). After that the JAR files could be removed, or replaced with empty files to show that they used to exist (with git filter-branch).
The manual work is proportional to the number of branches and merges. It would probably take a couple of days to do it all. I'll make a copy of the repository and have a closer look at it some day later when I have more time.
On Tue, Feb 28, 2012 at 2:50 PM, Esko Luontola <esko.luont...@gmail.com> wrote:
> On Tuesday, 28 February 2012 23:38:20 UTC+2, Bill Venners wrote:
>> Actually the problem was the *size* of the history. ScalaTest started >> as a "port" from the Java test framework SuiteRunner, which was >> started in 2001, thus it still had jar files checked in. So >> ScalaTest's history includes a lot of jar files, and that is >> impractical import into git. I did try it before deciding to move to >> Google Code, and it never completed the import.
> Did you try importing on your own machine, or using GitHub's import service? > The former should be more reliable. Even more reliable is if you run the SVN > repository on localhost, to avoid network problems and make it a bit faster.
I was trying via the Github import service.
> After importing, it's possible to filter the history to remove the JAR > files.
OK.
>> Say, did you get a chance to look at the path traits? Any suggestions >> for improvements?
> I read the documentation. Looks good.
> The documentation about ignored tests says "Note that a separate instance > will be created for an ignored test". Does this mean that the fixture setup > will be executed, but then no tests are executed? If the ignored test is > followed by siblings, then it should be possible to execute one of them, to > avoid wasting the effort which was spent on running the fixture setup.
Good idea, and you're right again. I'll jot that down as a potential performance enhancement for the future. It does complicate the "one instance per leaf node" explanation of the execution model. I tried hard to make it easy for users to form an accurate mental model of what's going on, to help them reason about their code, so I like that the explanation is simple without exceptions like "one instance per leaf node, unless there are ignores with siblings, in which case it will keep executing until..."
> On Wednesday, 29 February 2012 00:08:15 UTC+2, Kevin Wright wrote:
>> I've also looked into this. The main problem with Scalatest is that the >> SVN branch structure is very much non-standard, so it's not particularly >> amenable to any automated conversion.
> Git is very flexible in that regard (though it would require more or less > manual work). The history can be modified to fix any weirdness, and even > merges can be added to histories from pre-1.5 SVN repositories using "Git > grafts".
> I can have a look at the branch structure and estimate how much effort would > be needed to import it to Git and make it clean.
Well I wouldn't bother until I clean it all up. Only a few branches will likely survive such a purge.
Also, now that the history is safely in Google Code, I could also possibly just actually develop via github repos, and then tag them back on Google Code once released. The problem I had before was that ScalaTest started out on java.net, and when Oracle bought Sun, they decided to migrate off of Collabnet, whose platform really was a pain to use actually. So that was a good decision, but it meant that my history was going away there so I need to find a new home for it. Long story but that took 3 painful months because of the trouble I had importing to github and a bug importing between subversion versions. After being told I'd need to pay $1000 to get Collabnet to make a zip file of my subversion files, I finally ended up importing from java.net to the artima server using an older subversion version, then upgrading subversion on the artima server for that repo to work around the subversion version bug, and finally I was able to import it from the artima server to Google Code.
In some way I'd like to get the project on Github because it is so much easier for people to clone, make tweaks, and contribute them back. As a next step I'll clear out the dead branches, sometime after 1.8 goes to RC probably.
Bill
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
Ah, I see. Yes that's true. On java.net they put you under trunk/app, because they also had the home page under /trunk. So that's in the history. After I moved to Google Code I probably moved things to /trunk, /branches, and /tags like I was used to.
On Tue, Feb 28, 2012 at 3:56 PM, Esko Luontola <esko.luont...@gmail.com> wrote: > On Wednesday, 29 February 2012 00:50:17 UTC+2, Esko Luontola wrote:
>> On Wednesday, 29 February 2012 00:08:15 UTC+2, Kevin Wright wrote:
>>> I've also looked into this. The main problem with Scalatest is that the >>> SVN branch structure is very much non-standard, so it's not particularly >>> amenable to any automated conversion.
>> I can have a look at the branch structure and estimate how much effort >> would be needed to import it to Git and make it clean.
> I had a quick look at the repository. Appears that during java.net the > source was under /trunk/app and now the source is under /trunk. The branches > have some non-standard structures, for example I see at least these styles: > /branches/*/app/, /branches/app/*/, /branches/*/ and also some weird > mistakes such as /branches/app/bv-1/app/ (branch inside another branch).
> It would be possible to clean up the history so that /app is removed from > the beginning (with git filter-branch), the branches are imported correctly > (with git filter-branch), and merges are added to the pre-1.5 SVN "merges" > (with git grafts and filter-branch). After that the JAR files could be > removed, or replaced with empty files to show that they used to exist (with > git filter-branch).
> The manual work is proportional to the number of branches and merges. It > would probably take a couple of days to do it all. I'll make a copy of the > repository and have a closer look at it some day later when I have more > time.
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
On Wednesday, 29 February 2012 02:06:52 UTC+2, Bill Venners wrote:
> In some way I'd like to get the project on Github because it is so > much easier for people to clone, make tweaks, and contribute them > back. As a next step I'll clear out the dead branches, sometime after > 1.8 goes to RC probably.
I'll see what I can do about cleaning the history. Just keep on committing to Google Code as normal. No need to remove the unused branches. Then when I finish cleaning up the history, it'll be a quick job to import the latest changes from Google Code and glue them together with the cleaned history. After would be a good time to move to GitHub.
On Tue, Feb 28, 2012 at 4:16 PM, Esko Luontola <esko.luont...@gmail.com> wrote: > On Wednesday, 29 February 2012 02:06:52 UTC+2, Bill Venners wrote:
>> In some way I'd like to get the project on Github because it is so >> much easier for people to clone, make tweaks, and contribute them >> back. As a next step I'll clear out the dead branches, sometime after >> 1.8 goes to RC probably.
> I'll see what I can do about cleaning the history. Just keep on committing > to Google Code as normal. No need to remove the unused branches. Then when I > finish cleaning up the history, it'll be a quick job to import the latest > changes from Google Code and glue them together with the cleaned history. > After would be a good time to move to GitHub.
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
On Mon, Feb 27, 2012 at 10:34 PM, Bill Venners <b...@artima.com> wrote: > Hi rssh,
> That looks interesting. From what I gather, you want to "annotate" > tests with a bit of code (in a DSL) that defines what fixture stuff > should happen before and after a test. Is that an accurate summary?
This is accurate summary of near 1/2 of functionality. I.e. full summary can be:
1. define what fixture stuff should happen before and after a test. 2. run group of tests in some sequential order. 3. prevent setting the state of fixture to different states at the same time in parallel and allow parallel execution of tests on the same state.
Let I describe use-case for which such type of tests is needed:
So, let we develop software which later deployed in multiple client location. For example -- let we build CRM system on top of relational database.
Usually, when we build our system, we start with empty database (which is our state0), than we have test which add 100 client records (which is our state1) than we have sequence of tests which can run in state1 and change nothing (state) than we check removing of one user, than we entry to our CRM information about deals for left 99 persons, than we try to remove user which have associated information ... etc.
I.e. we model life-cycle of client as sequence of database states. We can say that can exists 3 types of tests: - tests, which do transition change (add 100 user: start state(empty) finish state(justCreated100Users) - tests, which check something and change nothing Jon is in our db: (start state(justCreated100Users) change(nothing) parallel) - tests, which leave state after themself undefined remove random used.
Now, .let look on next use-case:
1. administrator set price of item X to x1 2. client bought one such item and receive invoice 3. administrator set price of item X to x2 4. client bought one such item and receive invoice 5. summary of invoices must be x1+x2
And imagine that we want to add one to set of our tests.
1. we must have state where exists record for item X and for client which bought this item and admin which set the price. As alternative, we can setup such state inside each tests, but this will be long. 2 may-be we introduce next state (which we know) where exists users and items and history of payments. May be we will want use it as basic for next test. (for example -- delete item X fro database, look at summary of invoices again)
Also note, that run this test inside one database transaction is possible only in simple cases, in real life we have interactions with other systems, etc. So fully create all environment inside each tests will not only cost time, but in some cases impossible.
> I'm guessing that "start state (<value>)" means that before the test > starts, the passed state should be setup. I'm not sure what "finish > state(WITH_ONE)" means. Can you clarify? Also, what does
test which change state to state(WITH_ONE). Example of such state -- adding 100 users.
> "change(nothing)" mean? And what about "parallel"?
tests which does not change nothing. (for example -- implemented via rollbacked transaction)
> With "parallel" do you want to mark those tests that can run in > parallel as opposed to those that can't perhaps?
Yes. How this implemented: we collect such group of tests in nested suites and override runNestedSuite to run group tests in given order. Tests which share same start state, change nothing and marked as parallel can be executed in parallel.
> As far as github. I have considered having a mirror over there of > major releases. Almost like a tag, that people could do pull requests > against. I'll check into how easy that is. The reason I didn't move > over to begin with when I moved to Google code from java.net is I > didn't want to lose all the history. But it would be nice if people > could fork a release on GitHub and submit a pull request. Especially > if there was some easy way for me to get that back into Google code.
Exist bidirectional bridge git-svn http://schacon.github.com/git/git-svn.html [it was too complex for me when I was have project with few branches, but whith one main branch its works]
> As far as Distributors go, I believe you can define your own > distributor that has the features you need. You'd need to downcast it > with a pattern match to your more specific type, but I think that > would be the way to do it. Why don't you try implementing that so I > can see what it looks like. It is not impossible that we could add > such a thing to ScalaTest, but I would want to make sure it is very > very well understood before it goes in. I need something like that in > the afterAll method, for example.
Thank's I will look.
> I'm not too keen on opening up registerTest and registerBranch. Those > are not generic enough to be lifecycle methods. For one thing, they > don't work on traits where tests are methods like Suite (and in the > future, the new Spec). For another example look at the new path > traits. Those methods don't work on them either. Instead what you've > done is actually the intended way to customize: you made some custom > style traits. They look like the built-in ones but support your > managed fixture feature.
Hmm, good. [I was afraid that translating DSL to fixture DSL layes is overkill]
> That said, the other thing I was wondering about is did you consider > just executing the DSL code inside the test body? Instead of:
> start state(INITIAL) finish state(WITH_ONE) > it should "be able to add user " in { db => > //... > }
> It might be something like:
> it should "be able to add user " in { > withFixture(start state(INITIAL) finish state(WITH_ONE)) { db => > // ... > } > }
> This withFixture method you could place in a trait and users could mix > that into any style trait, even path traits or traits with test > methods, JUnit, TestNG, and so on. A lot more general. This probably > doesn't cover the parallel thing, but that depends on what you were > trying to achieve there. But I think it could take care of the setup > and teardown stuff if I'm understanding correctly what you're trying > to do.
Hmm, this can be useful for use-cases, when order of evaluation is unimportant, i.e. loading of state is cheap. I will think about real-life example.
> On Sun, Feb 26, 2012 at 11:09 PM, rssh <ruslan.s.shevche...@gmail.com> wrote:
>> Good day, colleagues.
>> I want to announce managed fixture -- reimplementation of scalatest >> fixture tests to situation, where our fixture have external state >> (like relational database) and test author can >> - mark each test with description of state preconditions and changes >> in simple DSL >> - provide functions for loading given state.
>> Then test engine can split all tests into parts, which require same >> state and can be executed in parallel, then execute this parts in >> order of execution, which minimize state loading. I.e. tests can be >> written in next style:
>> <pre> >> class MyFlatSpec extends managedfixture.[DBFixtureStateTypes.type] >> { >> val fuxtureStateTypes = DBFixtureStateTypes >> val fuxtureAccess = DBFixtureAccess >> import DBFixtureStateTypes.States._
>> behavior of "My datababase"
>> start state(INITIAL) finish state(WITH_ONE) >> it should "be able to add user " in { db => >> ..... >> }
>> start state(DATASET1) change(nothing) parallel >> it should "retrieve user with name Jon in our test dataset" in { db >> => >> inTransaction { >> val x = db.selectUser("Jon").headOption; >> assert(x!=None) >> assert(x.name == "Jon") >> } >> } >> </pre>
>> It's now in experimental stage: works at first glance but not yet >> fully tested.
>> In future I want to submit all into one of next version of scalatest, >> if this is possible. >> So, question - if this possible (?), and if yes -- how to do this >> (?) >> [May be mirror on github for pull requests ?]
>> Also it would be good to see some changes in next scalatest core:
>> - now managed-fixture suites contains reimplementations of each >> DSL. This can be simplicifed if scalatest core would allows hook >> arround registerTest .. registerBranch calls in fixture suite.
>> - distruptors - we can't load different database states at the >> same time from different tests, so we need from distruptors ability to >> run set of tasks inside external lock or may-be just callback call >> when all tests in distruptors are finished [?] >> // As alternative, we can implement own parallel execution scheme, >> does not depend from distruptors.
>> -- >> You received this message because you are subscribed to the Google >> Groups "scalatest-users" group. >> To post to this group, send email to scalatest-users@googlegroups.com >> To unsubscribe from this group, send email to >> scalatest-users+unsubscribe@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/scalatest-users?hl=en >> ScalaTest itself, and documentation, is available here: >> http://www.artima.com/scalatest
> -- > You received this message because you are subscribed to the Google > Groups "scalatest-users" group. > To post to this group, send email to scalatest-users@googlegroups.com > To unsubscribe from this group, send email to > scalatest-users+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/scalatest-users?hl=en > ScalaTest itself, and documentation, is available here: > http://www.artima.com/scalatest
Very interesting. I think I see where you want to go now. Nevermind what I said about withFixture for the time being. After reading your latest email, I think this sounds more like a job for overriding runTests and a creative application of tagging.
ScalaTest tests can be tagged in any style trait, and those tags can be accessed and used inside any lifecycle method (including runTests). My intuition is that there could be a way to tag tests to indicate the required starting state and the ending state, and whether or not it can run in parallel with others that have the same starting state. (Maybe you could make a simplifying assumption that any tests that require the same starting state and do not change the state can run in parallel.)
Anyway, to do this the user would need to define tags for all their starting and ending states, including one for the "NoChange" ending state. (Or, maybe if they don't specify an end state you could interpret that to mean NoChange, but that sounds error prone and less obvious to me.)
Then, you give them a way using your DSL to specify the order to perform the tests in based on their tags. Or maybe you could give them a way just to say the starting state, and use some algo to find an optimum way through the tests given the state changes. Let's call this the "script."
Lastly, override runTest to read the script and the tags, and figure out what order to run tests in, and run them in that order. As far as running in parallel, you're right it would be nice to be able to use the distributor if it is passed in, but you could also just fire up some threads and run in parallel yourself. Then you could do a countdown latch and figure out when to move on. But it does seem to me that if a Distributor is passed, its pool of threads should be used. And if a Distributor is *NOT* passed, maybe that means you should do those things sequentially anyway. Perhaps. Maybe you want to always do things in parallel that can, and if a distributor isn't passed you use your own.
But regardless, the upshot is that I think if you took this approach, you'd have a general library that could be used with any strait including ones where tests are methods. You may need some enhancements from ScalaTest in how tags are done and/or the Distributor. Might be worth coding up to see what you discover is actually needed from ScalaTest.
Bill
On Wed, Feb 29, 2012 at 3:30 PM, Ruslan Shevchenko
<ruslan.s.shevche...@gmail.com> wrote: > On Mon, Feb 27, 2012 at 10:34 PM, Bill Venners <b...@artima.com> wrote: >> Hi rssh,
>> That looks interesting. From what I gather, you want to "annotate" >> tests with a bit of code (in a DSL) that defines what fixture stuff >> should happen before and after a test. Is that an accurate summary?
> This is accurate summary of near 1/2 of functionality. I.e. full > summary can be:
> 1. define what fixture stuff should happen before and after a test. > 2. run group of tests in some sequential order. > 3. prevent setting the state of fixture to different states at the > same time in parallel and allow parallel execution of > tests on the same state.
> Let I describe use-case for which such type of tests is needed:
> So, let we develop software which later deployed in multiple client location. > For example -- let we build CRM system on top of relational database.
> Usually, when we build our system, we start with empty database (which > is our state0), than > we have test which add 100 client records (which is our state1) than > we have sequence of > tests which can run in state1 and change nothing (state) than we check > removing of > one user, than we entry to our CRM information about deals for left > 99 persons, than > we try to remove user which have associated information ... etc.
> I.e. we model life-cycle of client as sequence of database states. > We can say that can exists 3 types of tests: > - tests, which do transition change (add 100 user: start > state(empty) finish state(justCreated100Users) > - tests, which check something and change nothing > Jon is in our db: (start state(justCreated100Users) > change(nothing) parallel) > - tests, which leave state after themself undefined > remove random used.
> Now, .let look on next use-case:
> 1. administrator set price of item X to x1 > 2. client bought one such item and receive invoice > 3. administrator set price of item X to x2 > 4. client bought one such item and receive invoice > 5. summary of invoices must be x1+x2
> And imagine that we want to add one to set of our tests.
> 1. we must have state where exists record for item X and for client > which bought this item and admin which set the price. As > alternative, we can setup such state inside each tests, but this will > be long. > 2 may-be we introduce next state (which we know) where exists users > and items and history of payments. > May be we will want use it as basic for next test. (for example -- > delete item X fro database, look at summary of > invoices again)
> Also note, that run this test inside one database transaction is > possible only in simple cases, > in real life we have interactions with other systems, etc. So > fully create all environment inside each tests > will not only cost time, but in some cases impossible.
>> I'm guessing that "start state (<value>)" means that before the test >> starts, the passed state should be setup. I'm not sure what "finish >> state(WITH_ONE)" means. Can you clarify? Also, what does
> test which change state to state(WITH_ONE). Example of such state -- > adding 100 users.
>> "change(nothing)" mean? And what about "parallel"?
> tests which does not change nothing. (for example -- implemented > via rollbacked transaction)
>> With "parallel" do you want to mark those tests that can run in >> parallel as opposed to those that can't perhaps?
> Yes. How this implemented: we collect such group of tests in nested > suites and > override runNestedSuite to run group tests in given order. Tests > which share same > start state, change nothing and marked as parallel can be executed in parallel.
>> As far as github. I have considered having a mirror over there of >> major releases. Almost like a tag, that people could do pull requests >> against. I'll check into how easy that is. The reason I didn't move >> over to begin with when I moved to Google code from java.net is I >> didn't want to lose all the history. But it would be nice if people >> could fork a release on GitHub and submit a pull request. Especially >> if there was some easy way for me to get that back into Google code.
> Exist bidirectional bridge git-svn http://schacon.github.com/git/git-svn.html > [it was too complex for me when I was have project with few branches, > but whith one main branch its works]
>> As far as Distributors go, I believe you can define your own >> distributor that has the features you need. You'd need to downcast it >> with a pattern match to your more specific type, but I think that >> would be the way to do it. Why don't you try implementing that so I >> can see what it looks like. It is not impossible that we could add >> such a thing to ScalaTest, but I would want to make sure it is very >> very well understood before it goes in. I need something like that in >> the afterAll method, for example.
> Thank's I will look.
>> I'm not too keen on opening up registerTest and registerBranch. Those >> are not generic enough to be lifecycle methods. For one thing, they >> don't work on traits where tests are methods like Suite (and in the >> future, the new Spec). For another example look at the new path >> traits. Those methods don't work on them either. Instead what you've >> done is actually the intended way to customize: you made some custom >> style traits. They look like the built-in ones but support your >> managed fixture feature.
> Hmm, good. [I was afraid that translating DSL to fixture DSL layes > is overkill]
>> That said, the other thing I was wondering about is did you consider >> just executing the DSL code inside the test body? Instead of:
>> start state(INITIAL) finish state(WITH_ONE) >> it should "be able to add user " in { db => >> //... >> }
>> It might be something like:
>> it should "be able to add user " in { >> withFixture(start state(INITIAL) finish state(WITH_ONE)) { db => >> // ... >> } >> }
>> This withFixture method you could place in a trait and users could mix >> that into any style trait, even path traits or traits with test >> methods, JUnit, TestNG, and so on. A lot more general. This probably >> doesn't cover the parallel thing, but that depends on what you were >> trying to achieve there. But I think it could take care of the setup >> and teardown stuff if I'm understanding correctly what you're trying >> to do.
> Hmm, this can be useful for use-cases, when order of evaluation is unimportant, > i.e. loading of state is cheap. I will think about real-life example.
>> Thanks. Cool stuff.
>> Bill
>> On Sun, Feb 26, 2012 at 11:09 PM, rssh <ruslan.s.shevche...@gmail.com> wrote:
>>> Good day, colleagues.
>>> I want to announce managed fixture -- reimplementation of scalatest >>> fixture tests to situation, where our fixture have external state >>> (like relational database) and test author can >>> - mark each test with description of state preconditions and changes >>> in simple DSL >>> - provide functions for loading given state.
>>> Then test engine can split all tests into parts, which require same >>> state and can be executed in parallel, then execute this parts in >>> order of execution, which minimize state loading. I.e. tests can be >>> written in next style:
>>> <pre> >>> class MyFlatSpec extends managedfixture.[DBFixtureStateTypes.type] >>> { >>> val
