Using listeners to interact with Robot Framework

[Joseph Lorenzini]

Hi all,

I was discussing an issue in the users group and  I was told something that I'd like to verify the RF developers. 

I have a listener that runs at the end of each test case. The listeners imports the builtin module and then tags the test case with a tag based on test case status. 

See here for example code:

https://gist.github.com/jaloren/651978a1c157051a7fb3

This in fact works. However, I was told in the user group that listeners are never meant to interact with the robot framework and that the fact this worked was an accident.

There is no documentation in the user guide about this one way or the other. Should listeners be used to interact with the robotframework or not? If not, then I am thinking the documentation should be updated to reflect that minimally.

Personally, I think that would be a useful feature enhancement but  I obviously don't know the full ramifications of that kind of change. :)

Thanks,
Joe

[Bryan Oakley]

I definitely think listeners should be able to access BuiltIn(), or something similar. I was pleasantly surprised to find out I could access BuiltIn(), and like you I rely on this feature for some debugging tools I've written. I don't like relying on a feature working by accident either. I wish robot provided more robust introspection and runtime control. Since it doesn't, we're left with having to do tricks like calling BuiltIn() from within a listener.

I really hope they don't remove that feature, unless it's replaced with a supported API that lets us call any keyword from a listener.

--
You received this message because you are subscribed to the Google Groups "robotframework-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-d...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-devel.
For more options, visit https://groups.google.com/d/optout.

[Joseph Lorenzini]

Hi Bryan,

I strongly agree though I suspect the response from the devel team is: if you want  it, then submit PR for it. :) And I can't really argue with that. I bet the reason it doesn't exist is that no one cared enough to write the code.

Word of warning on using the builtin in a listener -- I ran into a highly undesirable behavior. I don't think i can call it a bug since its an unsupported functionality. 

I have a custom RF library that is used both as a remote library and as a regular python module imported into the framework. I was using the RF logger in that library. The logger does not work obviously (no handlers) if its a remote library since its not running in RF.

I needed the library to determine if it was running in RF or not and if it was not initialize handlers for the logger. I searched high low for a simple function call to do that but alas it does not exist. Tatu suggested using the Builtin module to figure that out. Basically if calling a builtin function throws RobotNotRunningError, then i know I am not in RF else I am not. See here for code from Tatu.

https://gist.github.com/jaloren/260efc5d0782da490a52

Annnd that worked fine. However, if I also have a listener that imports the builtin module, then this code (see above) throws the RobtNotRunningError even when the library is in fact running inside the framework.

I ultimately worked around the problem by checking whether this attribute is false or not: robot.running.context.GLOBAL_VARIABLES. I figured that the RF will not have global variables if its not running and that seems to work okay.

Still word of warning -- if builtin is used in a listener and in a custom library you should expect some odd behavior.

Joe

[Pekka Klärck]

2015-12-03 14:58 GMT+02:00 Joseph Lorenzini <jal...@gmail.com>:
> Hi all,
>
> I was discussing an issue in the users group and I was told something that
> I'd like to verify the RF developers.
>
> I have a listener that runs at the end of each test case. The listeners
> imports the builtin module and then tags the test case with a tag based on
> test case status.
>
> See here for example code:
>
> https://gist.github.com/jaloren/651978a1c157051a7fb3
>
> This in fact works.

I don't see anything wrong with that. Accomplish this is a lot easier
with listener v3 introduced in RF 3.0, though:

ROBOT_LISTENER_API_VERSION = 3

def end_test(model, result):
result.tags.add('PASSED' if result.passed else 'FAILED')

> However, I was told in the user group that listeners are
> never meant to interact with the robot framework and that the fact this
> worked was an accident.
>
> There is no documentation in the user guide about this one way or the other.
> Should listeners be used to interact with the robotframework or not? If not,
> then I am thinking the documentation should be updated to reflect that
> minimally.
>
> Personally, I think that would be a useful feature enhancement but I
> obviously don't know the full ramifications of that kind of change. :)

It's true that what listeners can do isn't specified that clearly. The
main problem there is that they can do so much. It is also true that
internal changes may change what listeners can do, but we try to avoid
such changes especially in minor versions. With major versions larger
changes are possible, but most that you can do via BuiltIn ought to be
safe anyway.

Notice also that if you want to alter execution results, using the
aforementioned listener API v3 is typically the easiest and safest
solution.

Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org