Testerman setup question
sarvi
have followed the instructions in the install.
This just aheadsup. I am trouble shooting.
Question:
I looking to setup a test framework for testing our product for a
group of developers.
There is a stack of UUT products that a developer can reserve and use
for testing.
Development happens on the linux machines where the sources for a
product get downloaded for development, build and testing.
Each developer gets his own Virtual Server for development.
I want the product/feature test code to be part of the sources pulled.
I want the developer to be able to build and modify tests on this
development machine.
I want him to be able to run the test cases individually or as a
suite.
He might run the test case under ipython interactively during
development running it step by step.
The framework would be telneting/sshing into the product.
The product will also run some testerman probes under the control of
the test cases running on the development machine.
Now to the question. I am not sure what setup Testerman should be run
in. It seems to require some central Testerman server running, with
clients on other machines. I got the probes part.
How would you deploy testerman in this setup for the above
requirement.
Sebastien Lefevre
Le 02/10/2010 12:56, sarvi a �crit :
> I am trying startup testterman on my server and it fails to startup. I
> have followed the instructions in the install.
> This just aheadsup. I am trouble shooting.
>
Could you elaborate on the problem you have ? Do you have a particular
error when running the "start all" testerman-admin command ? or it just
fails silently ? something else ?
The standard startup sequence starts two daemons:
- the testerman server, listening by default on tcp/8080,8081,8082,8083
- the testerman agent controller [server] (tacs), listening by default
on tcp/8087 and tcp/40000
You may check that these ports are available, or tune
conf/testerman.conf accordingly.
What is the output of "bin/testerman-admin status" ?
> Question:
> I looking to setup a test framework for testing our product for a
> group of developers.
> There is a stack of UUT products that a developer can reserve and use
> for testing.
> Development happens on the linux machines where the sources for a
> product get downloaded for development, build and testing.
> Each developer gets his own Virtual Server for development.
> I want the product/feature test code to be part of the sources pulled.
> I want the developer to be able to build and modify tests on this
> development machine.
>
> He might run the test case under ipython interactively during
> development running it step by step.
>
> The framework would be telneting/sshing into the product.
> The product will also run some testerman probes under the control of
> the test cases running on the development machine.
>
> Now to the question. I am not sure what setup Testerman should be run
> in. It seems to require some central Testerman server running, with
> clients on other machines. I got the probes part.
>
Testerman has been designed as a server-centric solution and focus on
resources sharing between users.
It encourages the use of a central repository for all scripts, however
it is quite possible to run scripts that were managed another way.
The testerman CLI client (bin/testerman), for instance, is fully able to
run a locally-provided ATS, and if your ATS depends on additional
modules, it can run what testerman calls a "(testerman) package" (an
archive containing multiple scripts and all their dependencies, quite
convenient if you want to send a campaign, for instance, from one site
to another one).
In this case, it is currently up to you to prepare the package before
running it through the CLI client.
However, the tests are still executed by and on the server and not locally.
This is because it shares a second kind of resources besides the
repository: test adapters, implemented in Testerman as what it calls
"probes".
When running an ATS, the ATS is "compiled" into a TE (Test Executable)
by the server that subsequently runs it as an independent process.
However, this process still connects to the server to send its log
events and to the TACS to be able to control probes that have been
deployed on remote agents (pyagents).
These agents are shared for all the users of the server, so that if
multiple users needs to control a probe that must run from a particular
location, they don't have to deploy their own agent each time.
Probes, however, don't have to always be remotely controlled. The TE may
also use "local probes" that are instantiated in its own process and
thus do not require a TACS. This is useful for simple cases where you
can afford to connect to the SUT from the server's IP address, but this
does not work anymore if, for instance, you need to watch a log file on
a remote machine or access anything that is not interfaced through the
network (sure, it may be done via ssh if the machine is ssh-reachable,
but this is underperforming to perform a tail-like command).
Anyway, what you'd probably need is a way to get (or create) the TE on
your workstation, then run the TE from it, probably without the need of
a server, i.e. only using local probes and the TTCN-3 primitives the
framework bring.
Runtime logs could be either sent to the server, or just locally
generated for offline analysis.
This is currently not something Testerman can do, but it can be planned.
A workflow like:
1/ from an ATS (+ dependencies) stored on workstation W, ask the server
to generate you a TE
2/ get the TE back on W
3/ execute the TE on W, optionally providing the server's IP address to
send logs and/or to control remote probes. Logs are also locally
generated and can the be opened in QTesterman (qtesterman --log <file>)
or anything else.
This execution model would have its own pros and cons, but it's not
without being interesting.
Pros: the TE can be run in a local debugger or interactive interpreter.
This is not the case when running from the server.
Cons: no delayed execution/scheduling, some local probes may not work
from your workstation (if missing dependencies that the server fulfills
at system level; for instance the ssh probe won't run from a windows
machine, the sql.oracle probe requires Oracle Instant Client libs that
are usually installed on the server, the ldap probe needs python-ldap, etc)
If it sounds OK to you, I can work on a way to implement these steps
(this is, basically, just a matter of packaging the TE to something
simple - tgz ? python egg ?) into the testerman CLI client first.
QTesterman is unlikely to be adapted to such a call flow (it is
designed, at the core, to server-centric).
> I want him to be able to run the test cases individually or as a
> suite.
>
Currently running an ATS runs all the test cases defined in the test
control.
How would you like to select the cases to run ?
thanks,
-seb