Re: Test Competition - Queries

[Matt Heusser]

I thought these questions, from Anindita Chandra, were great, and I should share the answers on the discussion list --

Part I, Correct Details and Summary, 'Reflecting my understanding back to the customer':

I have couple of queries in regard to the Test Competition and would like you to clarify them for my team - TeamIndia.

We are aware of the fact, that we will be given one application, one performance tool , one defect tracking tool for the test competition.

Our understanding is that the test competition is held on 2 days - one for function testing  and another round for performance testing.

Also went through the testing blog and  understand that the first round will be held on 21st April Eastern 10:01 AM and second round on performance testing over one weekend and the duration will be of 3 hrs.

Part II, Questions -

Would like to have responses to the following queries - 
Q1. what is the duration of the Test that will be held for functional testing.. I suppose 21st APril 10:01 AM EST  is the scheduled test start time? (i.e, will be 21st April 6:00 PM IST)

A1: Correct.  You have three hours to read the rules, talk to the customer, test, file bugs, write and file a test report.  The test report will be sent to an email address, you can use any reasonable document format. (.doc, .pdf, .pages, etc.)

Q2. Before testing , when would we get the chance to get a hang of the requirements and the testing objective and how will that be provided to the teams?

You get three hours.  As a special bonus because you asked nice, I may put the post 'up' 15 to 30 minutes early with the details of the competition, so teams can discuss and strategize.  Please don't start testing until the exact time; I will say "go" on the broadcast video we link to.

3. Do we require any specific setup on our machine that is pre-requisite for us to be able to participate in this test? Will the application to be tested accessible via web? If yes, does it require any prerequisite setup configuration in our machine. If not, How will be the access to the application made available to us for testing?

It will be entirely testable by web-based applications, thought you may want a variety of browsers and some mobile devices - or simulators.

4. When can we have access to the Defect Management tool so that we can have a hang of the tool for logging defects?

You should have it now! If you haven't been notifed of your login information, email myself and CC Smita Mishra - Sm...@QAZone.in. The only reason to not have it at this point is late registration, and my notes indicate that "Team India" did meet the registration deadline. 

5. Do we require to follow any process , standards or guidelines or have our own for all the testing activities.

The day of the competition I'll have a blog post with rules, what to test, and a URL to follow for a chat session with me, the "Customer" - you can ask questions of the customer to determine exactly how you'd like to file bugs and what to put in the report, but yes, we do want to offer a great amount of freedom in your process.

6. What is the schedule of  Round Two - Performance Test Competition?

You get the entire weekend (well, as much of it that you want to take) to plan and report on the performance, but you only get three hours to execute your performance tests.  We only have one test environment, you'll need to 'reserve' a 3-hour block of time if you would like to participate in this portion.  If you don't have a performance test tool, #NRGlobal has offered it's tool in the cloud, let me know, we'll send you the howto guide.  All you should need for that is a web browser and some standard microsoft (free) tools.

GREAT Questions!  Thank you for asking, what's next? :-)

--heusser

[Matt Heusser]

One important typo: The functional portion is all on April 19th.  Friday the 19th Eastern USA Time.

Would like to have responses to the following queries - 

Q1. what is the duration of the Test that will be held for functional testing.. I suppose 21st APril 10:01 AM EST  is the scheduled test start time? (i.e, will be 21st April 6:00 PM IST)

A1: Mostly Correct - the one issue is the date; the post will go up on the website around 9:30-9:45 Eastern time and the competition will begin at 10:00AM Eastern ON THE 19th.

That is the 19th.  The 19th.

You have three hours to read the rules, talk to the customer, test, file bugs, write and file a test report.  The test report will be sent to an email address, you can use any reasonable document format. (.doc, .pdf, .pages, etc.)

--

Matthew Heusser,

Principal Consultant, Excelon Development

http://www.xndev.com

Go to the event - Test Retreat, August 24, Madison, Wisconsin! testretreat.eventbrite.com

Or in Europe - Agile Team Academy, September, Amsterdam http://www.agileteamacademy.com/

[Matt Heusser]

Aha.  The 21st date came from a typo on this blog page:

http://www.nrgglobal.com/general/test-competition-update

I have updated it.  Thank you Nick Stefanski for pointing out the mistake.

--heusser

[Anindita Chandra]

Hi Matt,

 

Thank you Matt for the answers. It answered almost all our queries -

 

Ofcourse, We do have come with some more queries :-

 

Queries on Performance Test part

 

1. We would like to go for an open source tool? Can we go for the open source tool like Jmeter for the Performance test? 

 

2. Can you provide the 3 hr Available slots? so that we can reserve and block for the performance test execution.

3. When can we get to know about the Performance Requirement?

4. What are the Test deliverables expected for Performance Test?

What type of Reports are you looking at as part of the Performance test ? Will Perf the Tool report do or are you looking for analysis and reporting from Application and Database Logs?

 

Query on the functional Test Part

1. One clarification on the functional test part on the simulators, do you mean Mobile Simulators? Will that be made available to the Teams?

 

Thanks,

Anindita/Vivek

TeamIndia

[mheusser]

Answers inline!

On Apr 16, 8:49 am, Anindita Chandra <anindita.chan...@gmail.com>
wrote:

> Hi Matt,
>
> Thank you Matt for the answers. It answered almost all our queries -
>
> Ofcourse, We do have come with some more queries :-
>

> *Queries on Performance Test part*

>
> 1. We would like to go for an open source tool? Can we go for the open
> source tool like Jmeter for the Performance test?
>

You can use any tool you'd like. JMeter is fine. The app will be
entirely web based. If you are a little bit technical, I mean plain-
vanilla web based; it should run on TCP/IP on port 80.

>
> 2. Can you provide the 3 hr Available slots? so that we can reserve and
> block for the performance test execution.
>

Sure. We've got all friday evening free eastern right now, as well as
Sunday until 10PM. (Thought I believe I have another email from
someone asking about Friday early PM.)

>
> 3. When can we get to know about the Performance Requirement?
>

Friday at the standup video during the functional test portion.


> 4. What are the *Test deliverables* expected for Performance Test?

> What type of Reports are you looking at as part of the Performance test ?
> Will Perf the Tool report do or are you looking for analysis and reporting
> from Application and Database Logs?
>

I'm looking for a report to help me figure out the bottlenecks in the
system, what needs to be fixed, and if the software is good enough for
prime time. Exactly what goes in it, what graphs you generate,
summaries you write, or information you give is up to your team,
though we can talk a bit more Friday.

> *Query on the functional Test Part*

> 1. One clarification on the functional test part on the simulators, do you
> mean Mobile Simulators? Will that be made available to the Teams?
>

If you want to focus on laptop devices, you can get pretty far, but if
you want a mobile simulator, you will find some in this article I
wrote awhile back:

http://searchsoftwarequality.techtarget.com/tip/Defining-a-strategy-for-testing-mobile-devices

I hope that helps!

[Anindita Chandra]

 It helped..

 

Thank you

Matt.