AD Test policy

[Global QA Team]

Hulub, Lucian

Nov 23 2011

One of the initiative on the QA transformation Roadmap, is creating a test policy that wil govern QA activities. The test policy should be the document, we could present to customer when inquired about quality of our products, or present as reference to other departments in our relation with them. This document should not be longer that 1-2 text pages. When going into details, it should be backed up by our PDM/ISM and best practices as they are defined  around products and portofolio's.

Just a brief definition of a Test Policy:

"A Test Policy is a high level document and is at the top of the hierarchy of the Test Documentation structure. The purpose of the Test Policy document is to represent the testing philosophy of the company as a whole and to provide a direction which the testing department should adhere to and follow."

 

Please find the proposed document here: Test Policy Document

In the next two-three weeks, we are open to feedback and suggestions. As soon as we finilize the document the proposal is to have it signed by AD management.

[Global QA Team]

Laforge, Rose Marie

Nov 28 2011 in response to Hulub, Lucian

 Hi,

 

I would like to see the sentence " 

·        Root causes will be addressed either in the QA organization itself or facilitated in other areas of the company.

 

changed to something like this.  

 

 "Root causes will be addressed either in the QA organization along with the AD engineer or

facilitated in other areas of the company."

 

 We need to make sure the engineer who unit tested the software is involved in the Root cause anlysis along with QA. Just my opinion.

[Global QA Team]

Hulub, Lucian

Dec 5 2011 in response to Laforge, Rose Marie

 Thank you RoseMarie for your feedback.

The reason we have mentioned only QA organization in the first part of the sentence, is because we are referring to root causes which depend 100% on us or belong to our process, tools, testing approach, etc. Did the issues occur because of our inexperience, inappropriate use of tools, incompleteness of testplans/test specifications? Is a gap in requirements, or something that we missed? What can we do to address it ? Can we change anything to do better next time?

 

But that does not mean, that we can't ask help for engineering or ask for any other type of consultancies.

 

If the root cause is outside, we will facilitate the improvement, by providing them the data or the facts. We will stay accountable only for what is within our reach.

[Global QA Team]

Moore, Don

Nov 29 2011 in response to Hulub, Lucian

Thank you for pulling this together Lucian;

 

The definition of test stages is an excellent start, and an exercise that was needed in the last 4 projects I've been working with.  Can we please focus on this area and expand to the point that the policy can clearly identify the goal and scope of each testing stage? 

 

The policy might also consider the options for combining or skipping test stages on smaller projects. 

 

I ask this because even though the discussion is present the concensus of different people does not seem to get to a very clear end point.  We get "close enough" for one project then go on but each project can choose the scope and goals differently.

 

In each level, there is a question of when to stop and how much of the overall solution for a customer is involved.  Between Component, System and Solution, in particular there is potential for large overlaps and then to see that overlap and try to skip one or more of the levels.

 

I've seen Component done in a customer tailored environment with all interfaces and acquirers present, and also Solution done as a repeat of most of the component tests but using customer site equipment instead of simulators. If we skip Component and combine System/Solution, it may be more efficient but the scale and control needed may become unmanageable.

 

The unit test and user acceptance testing are much clearer, since the responsible parties are very close to the coding or production deployment respectively and have that to guide them. 

 

---- One other policy suggestion: 

 

Can we define the metrics to be collected in this process and how that feeds back into the cycle of new estimates?

 

We have a long tradition of project overruns on testing which is mostly due to optimistic estimate assumptions or contingencies not included in the estimates.  Both could be visible at the end of project when the estimate vs actual comparisons are done.

[Global QA Team]

Moore, Don

Nov 29 2011 in response to Moore, Don

 Further information -- looking at the QA handbook, there appears to be a lot of overlap between these documents, and the tests / roles defined.

 

Files in this community -- QA handbook_final.doc 

 

   The test levels there have specific objectives which answer many of the suggestions I had above.

 

   The question of metrics for effort estimated vs actual does not yet appear to be covered in that document. 

[Global QA Team]

Hulub, Lucian

Dec 5 2011 in response to Moore, Don

 Thank you Don for your feedback

 

That is correct. QA Handbook will be the document that will describe in detail about how we perform testing, or about our tasks and who is accountable for what.

 

the Test Policy, is a very high level document. Basically through this policy, the customer (could be real customer, or services, or engineering, or product management) is made aware about us, what drives/motivates us to test product, gives a clear indication about our accountability and determination, what we consider to be a minimum level of quality for our products. The test policy should be easy to read, both in terms of length and terminology.

 

On the metric side, this is good feedback Usually some of this information is present in the MTR. But definitely we can enhance what metrics we can track and put in the MTR.

 

We need to make the MTR's available to larger audience in QA, and easy to find. Potentially we can build a section here on this community, where we can attach the MTR's of finished project, from all portfolios. Every QA lead, then could add it here, and the entire QA team could peek inside it