Order of specs / execute once per test run

[Timothy Perrett]

Hey guys,

How can I setup specifications to be run in a specific order? Either that or how can i setup something to only one once for an entire test execution?

Cheers, Tim

[etorreborre]

Hi Tim,

Can you please tell which runner you're using and maybe a bit of
context?

> How can I setup specifications to be run in a specific order?

I think that you can run specifications in a specific order by
including them in that order in a "parent" specification.

> how can i setup something to only one once for an entire test execution?

Maybe you can use an object which will execute something once and set
a variable to avoid the same thing to be re-executed?

E.

[Timothy Perrett]

Hey Eric,

The issue is that I have a set of integration tests to test a Lift app and whilst they are logically different parts, I need to boot the lift environment first, so I was using the beforeSpec and afterSpec functions to setup the environment. However, depending upon the order the specs run, i occasionally get blow outs where the lift environment is already loaded.

What's the best way to handle this?

Cheers, Tim

> --
> You received this message because you are subscribed to the Google Groups "specs-users" group.
> To post to this group, send email to specs...@googlegroups.com.
> To unsubscribe from this group, send email to specs-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/specs-users?hl=en.
>

[etorreborre]

Funny I had exactly the same issue yesterday.

I'm surprised that this hasn't been solved earlier since this is bound
to happen with any test suite whether it's specs, ScalaTest, JUnit,...

I was planning on experimenting different kinds of solution:

- the clean way: check if there's a way to instantiate a Boot object
and ask it if it is instantiated before booting it. You know the
internals better than
me so if there was a way I guess you knew. A variation on that is
to check if one of the rules you add during the boot is active or not.

- the ugly way: do the boot everytime, catch the exception and
swallow it if it just says that the boot has been done

The first one is the best and if it's not possible right now maybe we
should create a ticket to expand the Lift API.

E.

[Timothy Perrett]

Hey Eric,

The only Lift api that exists is LiftRules.doneBoot; however, thats package private right now. I had played around with a hack that pretended to also be in lifts http package to access the variable, but that didn't really work. Your idea of checking one of the LiftRules is not a bad idea; again, its a little hacky though.

Yeah the ugly way is what I have right now, and man is it ugly ;-)

Cheers, Tim

[etorreborre]

Hi Tim,

I was about to create a new ticket when I realized that a quick
workaround would be to do the following:

class Boot {
def boot {
if (Boot.booted) return
... // the usual boot stuff goes here
Boot.booted = true
}
}

object Boot {
var booted = false
}

That's working for me.

E.

[Timothy Perrett]

Yeah, I also figured something like this - just stinks though because it means your production code has to have something pretty kludgy to accommodate testing. Maybe its just me, but it feels a little dirty; is it just me?

Cheers, Tim

[etorreborre]

You're right but you can still wrap that in a test-only object:

object BootTest {
var booted = false

def boot {
if (!booted) {
new Boot.boot
booted = true
}
}
}

E.

[Timothy Perrett]

Ironically, this is what I did before, however, it falls down if you have a mix of tests that only require the environment booted, and the full on integration tests that need the application booted within Jetty. Will have to have a think about it some more!

Cheers, Tim

[Timothy Perrett]

So I just had a play around, and there is a deeper issue. Its pretty irrespective of how you call boot, you'll always end up with issues with unchangeable state in LiftRules provided they are all within the same classloader.

What are your thoughts on stuffing my tests into wrapper traits and then composing them into one large runanble specification that does the setup and tear down? This is completely sub-optiomal, but really, I cant see a route out. If you have separate specifications (for example, I have some that need booted to use WebSpec and others that are selenium testing - distinctly different) even if they all use a full blown jetty which starts up and shuts down around each spec, the configuration of boot is still in the class loader, so it explodes on loading with the following spec. In addition, you never knew which will be the last spec, so you cant be sure where you need to call the cleanup operations.

Cheers, Tim

On 6 Mar 2011, at 00:30, etorreborre wrote:

[etorreborre]

> you'll always end up with issues with unchangeable state in LiftRules provided they are all within the same classloader.

I'd like to better understand this behavior. The "booted" variable
should prevent that we call the Boot class again. Is it because you
have some tests which start a jetty-instance and that instance boots
up Lift?

> What are your thoughts on stuffing my tests into wrapper traits and then composing them into one large runanble specification that does the setup and tear down?

If you want to go this route you'd better use specs2 :-). It is really
designed so that you can build examples and expectations from
different places and then associate them in the same specification.
I'm not too sure this would work ok with specs.

The last possibility is to define different test suites: unit test
suites where you control the Boot with BootTest and integration test-
suite where you boot "externally" once by starting your system under
test then run you run your tests. I've rarely seen both tests run in
the same suite.

E.

On Mar 6, 12:16 pm, Timothy Perrett <tperr...@gmail.com> wrote:
> So I just had a play around, and there is a deeper issue. Its pretty irrespective of how you call boot, you'll always end up with issues with unchangeable state in LiftRules provided they are all within the same classloader.
>
> What are your thoughts on stuffing my tests into wrapper traits and then composing them into one large runanble specification that does the setup and tear down? This is completely sub-optiomal, but really, I cant see a route out. If you have separate specifications (for example, I have some that need booted to use WebSpec and others that are selenium testing - distinctly different) even if they all use a full blown jetty which starts up and shuts down around each spec, the configuration of boot is still in the class loader, so it explodes on loading with the following spec. In addition, you never knew which will be the last spec, so you cant be sure where you need to call the cleanup operations.
>
> Cheers, Tim
>
> On 6 Mar 2011, at 00:30, etorreborre wrote:
>
>
>
>

> u

[Tim Perrett]

Interesting - I dont have time this second to write a larger
explanation, but I will tomorrow. I compiled into traits this morning
and its working great now. Certainly not ideal, but it'll do for now.

Cheers, Tim

[Timothy Perrett]

Hey Eric - some notes inline!

Cheers, Tim

On Sunday, March 6, 2011 9:25:39 PM UTC, etorreborre wrote:

> you'll always end up with issues with unchangeable state in LiftRules provided they are all within the same classloader.

I'd like to better understand this behavior. The "booted" variable
should prevent that we call the Boot class again. Is it because you
have some tests which start a jetty-instance and that instance boots
up Lift?

Thats right - for example, I need to start a jetty server to run selenium testing and also testing for the web services (using lift test kit). Additionally, you need to run the WebSpec tests with Boot initialised, so if you have a mix of ways to initialise that environment and either way you do it (either direct new Boot().boot or load jetty, when the other one tries to do it there will be an exception thrown). 

Subsequently, you cant stop jetty booting lift (it needs too as part of its initialisation process) so this is where the external variable falls down. To that end, I took the view that I would only boot jetty as its "fattest" and I had least control over it. 

 

> What are your thoughts on stuffing my tests into wrapper traits and then composing them into one large runanble specification that does the setup and tear down?

If you want to go this route you'd better use specs2 :-). It is really
designed so that you can build examples and expectations from
different places and then associate them in the same specification.
I'm not too sure this would work ok with specs.

Interesting - this is perhaps feasible in future, but not right at the moment as this is actually for Lift in Action so I have to be finish in the next couple of weeks. 

 

The last possibility is to define different test suites: unit test
suites where you control the Boot with BootTest and integration test-
suite where you boot "externally" once by starting your system under
test then run you run your tests. I've rarely seen both tests run in
the same suite.

Well, you cant strictly unit test any but the most trivial Lift code - practically everything requires part of the state stack initialised. I think hardly anyone uses test kit (for service testing) because its probably one of the least documented parts of Lift, which is why one never sees it. Without it though, there are very few ways to actually test anything defined by LiftRules.dispatch et al. 

 

E.

[Indrajit Raychaudhuri]

Hey Eric, Tim,

I am following this discussion with interest :)

Recently, had similar trouble in couple of lift-webkit specs as well when we moved to SBT. As last resort, got away by booting up two separate instances of embedded jetty (on two separate ports) of course, at the cost of performance.
However, I am still on the lookout for a more elegant way.

FWIW, these specs are quite old and they never were a problem in Maven environment because of forked execution and the way in which they were loaded (via JUnit suite).

Cheers, Indrajit

[Timothy Perrett]

Yeah multiple Jetties is one way, but it feels wrong as you said.

Eric, we wait with baited breath and hope you have some awesome ideas :-D

Cheers, Tim

[etorreborre]

Unfortunately I don't know the internals of lift well enough, but if
there's some static state somewhere, it should be possible to read it
before rebooting:

- by opening up the API (create a ticket for that)
- by subclassing the Boot class to provide an access to the lift
rules to know if they're booted
- by adding a "tag" rule or other boot artifact which does nothing
but can be accessed to know if the boot sequence was done or not

Why not following this discussion on the lift mailing-list? Lift users
must also have good ideas on this subject.

E.

[Derek Chen-Becker]

There is planned work to make LiftRules scoped so that you can have multiple independent instances in one VM. We're doing this specifically for the case of testing. It probably won't make it in, though, until after 2.3.

Derek