> Hi there - I'm trying to output a message to the console when our tests are
> running.

> We already have our Robot tests integrated with our CI server (TeamCity) by
> transforming the report XML which gives us pass/fail and counts of each but
> I'd like to extend this by reporting which tests are running, how long each
> takes, etc whilst the tests are executing.I can do this by outputing
> messages to the console by putting the following in the test setup and
> teardown respectively:

> ##teamcity[testStarted name='${TEST_NAME}' captureStandardOutput='true']
> ##teamcity[testFinished name='${TEST_NAME}']

> I've tried using the Log keyword with the level set to WARN, but then these
> messages will appear as errors in the log. Is there any way of sending raw
> output to the console without this happening (I don't really even need these
> in the log).

> Yep, thanks. Following Bryan's advice I put together a listener which only
> took a couple of minutes (that part of the docs must have been in a blind
> spot for me). TeamCity correctly reads in the tests (I now get details of
> test timing and the names of the tests) - but only reports them through the
> UI once the nant task I'm using to run the tests completes so it hasn't
> given me exactly what I was after. I'll enquire on the TeamCity forums to
> see what I'm missing (maybe I'm missing a NAnt option or perhaps I need to
> report the suits as well) as the Robot side seems fine now.

> Thanks for all the work you guys have put in - I've been using RF for over a
> year now and I'm a big fan!

> Cheers
> Finn

> On 16 September 2010 18:26, Pekka Klärck <p...@iki.fi> wrote:

>> 2010/9/16 Finn Neuik <fin...@gmail.com>:
>> > Hi there - I'm trying to output a message to the console when our tests
>> > are
>> > running.

>> > We already have our Robot tests integrated with our CI server (TeamCity)
>> > by
>> > transforming the report XML which gives us pass/fail and counts of each
>> > but
>> > I'd like to extend this by reporting which tests are running, how long
>> > each
>> > takes, etc whilst the tests are executing.I can do this by outputing
>> > messages to the console by putting the following in the test setup and
>> > teardown respectively:

>> > ##teamcity[testStarted name='${TEST_NAME}' captureStandardOutput='true']
>> > ##teamcity[testFinished name='${TEST_NAME}']

>> > I've tried using the Log keyword with the level set to WARN, but then
>> > these
>> > messages will appear as errors in the log. Is there any way of sending
>> > raw
>> > output to the console without this happening (I don't really even need
>> > these
>> > in the log).

>> There are at least two ways to do this:

>> 1) Implement a keyword that writes messages to stderr stream. These
>> messages are echoed to the console as well as written to the log.

>> 2) Implement a listener that prints this information. Outputs are not
>> captured when listeners are executed so whatever they print to stdout
>> or stderr will end up to the console directly. Listeners also have
>> information about how long a test took readily available. For more
>> information about listeners see:

>> http://robotframework.googlecode.com/svn/tags/robotframework-2.5.3/do...

>> Cheers,
>>    .peke

> I think a listener would be pretty generic - currently we're using XSLT to
> transform the test output into the results which TeamCity requires but
> looking at the listeners we can do everything using those instead. This
> would also have the advantage that I can remove the pause I need before
> doing the transform whilst waiting for the report to finishing spooling.

> I've come across a couple of issues - first of all the test results are only
> getting reported at the end of the test run rather than in realtime. I think
> is probably just something I'm missing in what TeamCity requires so should
> be easy enough to solve.

> The other is that some of our test names have characters which cannot be
> output to the windows console using the straight print statement due to code
> page issues, so I'll need to transform the output I guess.

> I'll try and solve these issues next week and share the results then.

> Cheers
> Finn - the world can't have enough Finns I think!

---------- Forwarded message ----------
From: Finn Neuik <fin...@gmail.com>
Date: 2010/9/23
Subject: Re: Output message to console
To: Pekka Klärck <p...@iki.fi>

Well I got it all working earlier this week - the problem seems to
have been the "[exec]" text which NAnt prefixes console output with.
As soon as I swapped to MsBuild everything worked first time. I'll try
and get something written up soon and get it over to you.

On 18 September 2010 12:46, Pekka Klärck <p...@iki.fi> wrote:

> [Replying also to the list.]

> 2010/9/17 Finn Neuik <fin...@gmail.com>:
> > I think a listener would be pretty generic - currently we're using XSLT to
> > transform the test output into the results which TeamCity requires but
> > looking at the listeners we can do everything using those instead. This
> > would also have the advantage that I can remove the pause I need before
> > doing the transform whilst waiting for the report to finishing spooling.

> Sounds good.

> > I've come across a couple of issues - first of all the test results are only
> > getting reported at the end of the test run rather than in realtime. I think
> > is probably just something I'm missing in what TeamCity requires so should
> > be easy enough to solve.

> Listeners allow real time updates so this sounds like an issue with
> TeamCity integration. Hope you can resolve it.

> > The other is that some of our test names have characters which cannot be
> > output to the windows console using the straight print statement due to code
> > page issues, so I'll need to transform the output I guess.

> Do you necessarily need to go through the console? Could you use files
> instead? Or does TeamCity have an API you could you directly from the
> listener? That would probably be the best solution.

Calling an API would be much better, but as far as I can tell to
communicate with teamcity you need to either write a custom runner
(which would be great, but I don't have the time for that at the
moment, especially as I now have the result I need), via an xml file
(which is how we were integrating until this week, but this only gives
results at the very end of the run) or via the console - which gives
realtime details about what's happening as the tests are running.

> > I'll try and solve these issues next week and share the results then.

> Cool! If you come up with something generic, are you interested in
> writing a wiki page about TeamCity integration? If there's some
> generic code, starting a separate project might be even better option.

> > Cheers
> > Finn - the world can't have enough Finns I think!

> Must agree with that! =)

> Cheers,
>    .peke
> --
> Agile Tester/Developer/Consultant :: http://eliga.fi
> Lead Developer of Robot Framework :: http://robotframework.org

> ---------- Forwarded message ----------
> From: Finn Neuik <fin...@gmail.com>
> Date: 2010/9/23
> Subject: Re: Output message to console
> To: Pekka Klärck <p...@iki.fi>

> Well I got it all working earlier this week - the problem seems to
> have been the "[exec]" text which NAnt prefixes console output with.
> As soon as I swapped to MsBuild everything worked first time. I'll try
> and get something written up soon and get it over to you.

> On 18 September 2010 12:46, Pekka Klärck <p...@iki.fi> wrote:

> > [Replying also to the list.]

> > 2010/9/17 Finn Neuik <fin...@gmail.com>:
> > > I think a listener would be pretty generic - currently we're using XSLT to
> > > transform the test output into the results whichTeamCityrequires but
> > > looking at the listeners we can do everything using those instead. This
> > > would also have the advantage that I can remove the pause I need before
> > > doing the transform whilst waiting for the report to finishing spooling.

> > Sounds good.

> > > I've come across a couple of issues - first of all the test results are only
> > > getting reported at the end of the test run rather than in realtime. I think
> > > is probably just something I'm missing in whatTeamCityrequires so should
> > > be easy enough to solve.

> > Listeners allow real time updates so this sounds like an issue with
> >TeamCityintegration. Hope you can resolve it.

> > > The other is that some of our test names have characters which cannot be
> > > output to the windows console using the straight print statement due to code
> > > page issues, so I'll need to transform the output I guess.

> > Do you necessarily need to go through the console? Could you use files
> > instead? Or doesTeamCityhave an API you could you directly from the
> > listener? That would probably be the best solution.

> Calling an API would be much better, but as far as I can tell to
> communicate withteamcityyou need to either write a custom runner
> (which would be great, but I don't have the time for that at the
> moment, especially as I now have the result I need), via an xml file
> (which is how we were integrating until this week, but this only gives
> results at the very end of the run) or via the console - which gives
> realtime details about what's happening as the tests are running.

> > > I'll try and solve these issues next week and share the results then.

> > Cool! If you come up with something generic, are you interested in
> > writing a wiki page aboutTeamCityintegration? If there's some
> > generic code, starting a separate project might be even better option.

> > > Cheers
> > > Finn - the world can't have enough Finns I think!

> > Must agree with that! =)

> > Cheers,
> >    .peke
> > --
> > Agile Tester/Developer/Consultant ::http://eliga.fi
> > Lead Developer of Robot Framework ::http://robotframework.org