Getting started with end to end testing - Use of Main class

[Richards Peter]

Hi experts,

I am reading the wonderful book - Growing Object Oriented Software Guided by Tests. It has motivated me to develop applications by writing end to end tests first.

I am trying to practice the concepts in the book by writing a client-server program (TCP server-client). Unlike the sniper application in the text, which is a client program, my application is a server program. I am trying to make my client connect directly to the server without using a communication layer like XMPP(Jabber). I am trying to use plain Strings as messages between Server and Client. I do not plan to have a UI in this application. I think I will have to assert, in my end to end tests, the behavior of Fake Clients during various server events.

I found that the authors are making use of a Main class for end to end tests. I am not sure how to use such a class for my use case.

class Main{

public static void main(String args[]){
  //Initialize server socket based on port in args[]
}
}

If I want to write some message to accepted clients in some test method(say announce server shutdown) or shutdown my server in the teardown method (closing server socket), I will have to expose the server socket. How can I best implement this concept in end-to-end test?

Is it fine if I do not use a Main class in my end to end test?

Thanks,
Richards Peter.

[Matteo Vaccari]

Hi Peter,

it's OK not to use a Main class.  A Main class might be useful

1. if you want your end-to-end test to be in a separate process with respect to the system under test; which might be useful, because you can then point your end-to-end test to any running instance of your system; 
2. and also because you ensure that all interactions between the test and the system are going through the public interface of your system, instead of "behind the scenes".
   

If you are getting the hang of how end-to-end tests work, it's simpler to start with just JUnit.  For instance, this is the end-to-end test that I'm proposing to my college students:

https://goo.gl/eZr43W

In your case, you could do something similar:

public MyTcpServer server;

@Before

public void startServer() throws Exception {

  this.server.start(SOME_PORT);

}

@After

public void stopServer() throws Exception {

  this.server.stop();

}

@Test 

public void testSomething() throws Exception {

   // pseudocode ... 

  Socket socket = new Socket("localhost", SOME_PORT);

  socket.getWriter().write("Hello");

  assertEquals("Goodbye", socket.getReader().readLine());

}

There is a very old article by Robert Martin that shows how to TDD a socket server.  

http://www.objectmentor.com/resources/articles/craftsman6.pdf

There is also a mention of how to TDD sockets in Kent Beck's "TDD By Example".

https://books.google.it/books?id=gFgnde_vwMAC&lpg=PP1&dq=tdd%20by%20example&hl=it&pg=PA135#v=onepage&q&f=false

Matteo

[Richards Peter]

Hi Matteo,

Thank you for sharing those links. I understood how to unit test Server class. In the example that you shared we unit test TcpServer class. 

I think I should explain my problem in detail.

In an end-to-end test we may have to start that server in the main() and wait for client connections (I am trying to adopt the approach taken by authors of the book). If we follow such an approach in my problem, we may lose the reference of TcpServer object(because we instantiate TcpServer within the main function) . We may not be able to call stopServer() from the end-to-end test class.

In the AuctionSniper example, we had a JFrame through which we could shut down the client connection (on close event). I am wondering how a similar functionality can be achieved in the TcpServer end-to-end test case.

Hope I have explained the problem in detail. I am trying to identify a better way to start and shut down my server from end to end tests (assuming that there will be a main function). My server will have to notify existing client connections before it shuts down. I am trying to test such a scenario in my end-to-end test. This is one reason why I asked whether a main class is mandatory to perform end-to-end test. Of course there are advantages,as you pointed out, when we start end-to-end tests through a main class.

Thanks,

Richards Peter.

[Matteo Vaccari]

You could have a script that starts the server, waiting for the server to be up, then starts the tests, and then shuts down the server.  

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steve Freeeman]

Don’t forget that the main() method is just another method. You can call it directly from your @BeforeClass method. Alternatively, you can make it just a thin wrapper around the real functionality and call /that/ in your test code.

The interesting question is how long it takes to start up your server. One of the effects of working this way is that it encourages keeping the application lightweight.

S

[Colin Vipurs]

I've tried this a few times in the past with varying levels of success.

As Steve has said, you can make main() a thin wrapper around your actual code, which is a good idea anyway.  The only problem with this is that it's possible to miss something that you don't find until you go to production.  This actually happened to me as I forgot to instantiate the run loop, but the ROI of covering that with a test wasn't worth it and it's the kind of mistake that only happens once.

I've also done what Matteo has said and used a script to start the server, but I made this part of the build system itself (I use Gradle so this was trivial) and had a single, simple end-to-end test to verify bootstrapping etc.  Every other "end to end" test was driven by the code the main() method was instantiating.  Moving forward I think this is an approach I'd use again as it "felt" better and allowed me to drive out the main() method with a test instead of it just magically appearing.

One thing to think about is how you will /cleanly/ shut down your server - are you trapping the kill signal or will it be more explicit, like a shutdown port? Is this the tests way of telling you that you're missing some functionality?

I'd also agree with Steve with keeping your application lightweight.  Lately I've been actively /trying/ to drive more tests end-to-end (I'll be talking at JAX London later in the year on this subject) and keeping app startup time as fast as possible.  This has the advantage, or disadvantage depending on your point of view, that heavyweight frameworks like Spring and Hibernate become a no-go area as they increase startup time too much.

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Maybe she awoke to see the roommate's boyfriend swinging from the chandelier wearing a boar's head.

Something which you, I, and everyone else would call "Tuesday", of course.

[Steve Freeeman]

On 15 Jun 2015, at 10:24, Colin Vipurs <zodi...@gmail.com> wrote:
> I've tried this a few times in the past with varying levels of success.

One of the tricky things about this area is that does take experience, because it depends so much on local circumstances

> I'd also agree with Steve with keeping your application lightweight. Lately I've been actively /trying/ to drive more tests end-to-end (I'll be talking at JAX London later in the year on this subject) and keeping app startup time as fast as possible. This has the advantage, or disadvantage depending on your point of view, that heavyweight frameworks like Spring and Hibernate become a no-go area as they increase startup time too much.

+1 on this. There are exceptions such as third-party systems or some high-performance stuff that has to set itself up in some way. Any other case is probably excess overhead.

S

[Władysław Mettler]

Hi

Bringing up the SUT may take some time. You can usually offset this by:

1. Make sure the system can be deployed in an automated/scriptable way (Chef, Ansible, bash scrip, whatever...)
2. Deploy the system to a virtual machine.
3. Take a snapshot.
4. Measure what takes longer, restoring the snapshot or deploying the system from scratch.

Additionally this gives you a clean starting point which you can use instead of 'undoing' the fixture. And you can take more snapshots during your test suite execution to have starting points after a specific fixture. And you can take snapshots on failures so that you can do a post-mortem. And you can use the same procedure in your continuous integration.

Vlad

[Israel Fonseca]

Vlad, I think that this snapshot approach would be very hard to maintain when the fixtures inside the snapshots had to change. What do you think about it?

--

[Władysław Mettler]

Hi,

Please note that this is a temporary snapshot intended for use between different test cases within the same test suite. It is discarded as soon as the test suite is done. The only intention there is to speed up the reset to the state defined as a starting point.

Vlad

--

---
You received this message because you are subscribed to a topic in the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/growing-object-oriented-software/gB1V71ZzEPY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to growing-object-oriente...@googlegroups.com.