Fail test when return value is false

[James]

Hi all,

I am working with a framework that currently also is executed using
fitnesse. In fitnesse to fail a test you return a boolean value, so
my library does this. Is it possible to get robot framework to work
in a similar way? I was looking at using a listener but that doesn't
seem to have access to the return value, or am I incorrect? Or can I
do something with the run_keyword method? I'm using java.

Thanks,
James

[James]

Sorry one last question. Is there also a way to pass null through if
you are expecting 3 arguments but only getting 2?

Thanks again,
James

[David]

RF formally uses exceptions within a test library (code) to indicate
failure. On the user/test level, if you wanted to fail based on false
return value, you would have to use comparison keywords in the BuiltIn
library like Should Be False, or Should Be Equal (with value equal to
false), etc. take a look at the BuiltIn library for details.

If you are writing a test library in code, you should try to throw
exceptions to report failure. Otherwise, the RF user / test writer,
will have to specifically check/compare against false return values.
If your library calls other libraries/code, then if you get return
value of false, then throw exception.

Looks like your best option right now is to add additional logic in RF
test to check return value with the BuiltIn library keywords to force
a failure if return value is indeed false.

[James]

Thanks for the reply David,

I was hoping that there would be a better way around than that. I
can't ask them to check after each keyword execution that the value
returned is true/false. So unless someone else has a way to extend
the keyword_run method or something similar then I guess I will just
have to change the keyword framework to throw exceptions.

Thanks,
James

[Janne Härkönen]

Hello James,

On Thu, Dec 1, 2011 at 2:59 AM, James <jamesf...@gmail.com> wrote:
> Thanks for the reply David,
>
> I was hoping that there would be a better way around than that.  I
> can't ask them to check after each keyword execution that the value
> returned is true/false.  So unless someone else has a way to extend
> the keyword_run method or something similar then I guess I will just
> have to change the keyword framework to throw exceptions.

In the long run, having to prefix all keywords with some kind of 'Run
Keyword ...' seems also onerous.

One thing you could do is to create a dynamic library [1] that runs
your existing keywords. Depending on how your current keywords are
structured, this might be easier than changing each keyword method. It
would be much easier to use Jython to create the front-end, but it is
possible also using pure Java.

The general idea in the dynamic library would be like this:

class MyLib:

def get_keyword_names(self):
# Somehow create a mapping of valid kw methods

def run_keyword(self, kw_name, *args):
kw = self._get_keyword(kw_name)
if not kw(*args):
raise AssertionError('Error message')

Now obviously the hard part is to figure out the valid keywords, but
if this seems like a worthwhile idea to you, we can continue with more
concrete examples.

[1] http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.6.3#dynamic-library-api

hth,
--J

--
Janne Härkönen | http://reaktor.fi
http://twitter.com/#!/janneharkonen

[James Farrier]

Hi Janne,

Have you seen any examples using Java?  This seems like a reasonable idea to me.

Thanks,

James

[Janne Härkönen]

On Thu, Dec 1, 2011 at 8:05 AM, James Farrier <jamesf...@gmail.com> wrote:
> Hi Janne,
>
> Have you seen any examples using Java?  This seems like a reasonable idea to
> me.

I have implemented this a couple of times with Java also.
Unfortunately, none of those implementations are open source.

There are some factors that affect what is the "easiest" way to
implement such a dynamic library:

1) do you have full control of the code you want to wrap ?
2) how many keyword methods will there be?
3) how often new keyword methods are added?
4) how often signatures of existing keywords methods change?
5) how heterogenic the signatures of current keyword methods are?

The last point has probably the biggest impact in the overall
complexity of the library. In Java, the signature of the runKeyword
method that Robot Framework uses is this:

public Object runKeyword(String kwName, Object[] args)

If all of the keyword methods have uniform argument types (i.e. they
only take Strings as arguments), mapping these from a Object[] is
pretty straightforward. If this is not the case, then some additional
casting is needed, and this might get complicated.

If you can estimate answers to above question, I might be able to whip
up some kind of examples/suggestion how I would proceed to solve this
problem. Without additional, the options are too numerous to enumerate
in a single email. (You also might have provoked me to write a blog
post about this :) )

thanks,

[James Farrier]

Hi Janne,

A blog posts sounds like something that would be useful, not only to me but also to I'm guessing a other people as they go through the same path.

As for your questions:

I have full control of the code I want to wrap.

There are currently 32 keyword methods, I also expect this to grow

Methods are added on an adhoc basic, usually 2 a week.

Existing methods don't change very often, you might get one change every couple of weeks.

The signatures aren't really heterogenic

Fortunately all of the keywords only take Strings as inputs, as that is how Fitnesse works.  However they can take a different number of arguments, so the user might enter 2 strings and the method takes 3, so I would like to be able to fill in that 3rd string without making a new wrapped keyword.

Thanks again,

James

[James]

Hi Janne,

Did you get anywhere on this? No worries if not, I will start work on
making my framework throw errors at false.

Thanks,
James

On Dec 2, 9:34 am, James Farrier <jamesfarr...@gmail.com> wrote:
> Hi Janne,
>
> A blog posts sounds like something that would be useful, not only to me but
> also to I'm guessing a other people as they go through the same path.
>
> As for your questions:
> I have full control of the code I want to wrap.
> There are currently 32 keyword methods, I also expect this to grow
> Methods are added on an adhoc basic, usually 2 a week.
> Existing methods don't change very often, you might get one change every
> couple of weeks.
> The signatures aren't really heterogenic
>
> Fortunately all of the keywords only take Strings as inputs, as that is how
> Fitnesse works.  However they can take a different number of arguments, so
> the user might enter 2 strings and the method takes 3, so I would like to
> be able to fill in that 3rd string without making a new wrapped keyword.
>
> Thanks again,
> James
>

> On 1 December 2011 20:21, Janne Härkönen <janne.t.harko...@gmail.com> wrote:
>
>
>
>
>
>
>
> > On Thu, Dec 1, 2011 at 8:05 AM, James Farrier <jamesfarr...@gmail.com>