Looking for some help/examples using robot api and parsing model

[Andrew Palmer]

Hi All,

I had a couple of questions regarding the robot apis and parsing model. A while back, I had built a tool that 'checks' robot files and verifies they are following a certain standard, such as test name format, keyword name format, checking unused variables, etc...

I recently started to refactor the checking tool after coming across the robot APIs as I found a lot of the parsing tools I developed had functionality already done in robot APIs.

Currently, for test suites, I built a class to make it a bit more readable and easier for our users, example as follows:

def __init__(self, suite):

        self.suite = TestData(source=suite)

        self.name = self.suite.name

        self.tests = [test.name for test in self.suite.testcase_table.tests]

        self.force_tags = [tag for tag in self.suite.setting_table.force_tags]

        self.suite_variables = [var.name for var in self.suite.variable_table.variables]

        self.keywords = [keyword.name for keyword in self.suite.keywords]

The above works great for test suite files, but I am looking to add the following functionality:

- Get the full header, (i.e. *** Settings *** instead of just Settings), not sure if this is possible? I know based on the table I can use .header

- Get the test case source, example, if I have:

My_Test

    Keyword     arg

    Another Keyword      arg

Just return the test source:

    Keyword     arg

    Another Keyword      arg

- Get all Library and Resource imports, so at the top of the file I would get any resource or python library I have imported. I saw there is some 'imports' within the settings_table but it doesn't seem to work as I had expected.

- I built in functionality for unused variables, just curious if something like this exists? Same for duplicate tests/keywords.

Last question is in regards to ResourceFile(). I used a very similar class as above:

    def __init__(self, resource):

        self.resource = ResourceFile(source=resource)

        self.name = self.resource.name

        self.keywords = [keyword.name for keyword in self.resource.keyword_table.keywords]

name works, but keywords does not. I tried a few other methods for getting keyword names, such as resource.keywords (which based on dir(resource) seems like a valid option). I get the exact same problem when trying to get resource variables.

I appreciate the help. I've been going through the robot docs and such for some time but I feel that some areas are missing examples and functionality, a bit hard to read. I was also a bit confused on difference between using the different classes between api and parsing, many seem to achieve the same functionality.

Thank you.

[Bryan Oakley]

If you're writing a robot syntax-checking tool, are you aware of robotframework-lint (https://github.com/boakley/robotframework-lint)? Using it might let you focus more on the rules and less on the parsing. I think it will let you do everything you mentioned (get header info, get raw source, etc). 

It has all of the parsing and tree-traversal code built in, all you have to do is write the code for the rules you want to enforce. It uses a custom parser since the built-in parser throws away line number information. The parser returns objects that are much easier to work with than the ones provided by the official parser. This tool only supports the plain text formats (tsv, space, and pipe formats) not HTML or reStructuredText)

If you have it installed, you can also use the parser to write your own tools. An example of using the parser to traverse all of the objects in a suite are on its wiki: https://github.com/boakley/robotframework-lint/wiki/Using-the-parser-to-walk-a-suite.

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at https://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[Joseph Lorenzini]

Hi Andrew,

What Bryan said. But also, I find the simplest solution sometimes works best -- I just robot framework API to convert the files into CSV, use the python CSV file to parse them, and then convert them back to robot framework files. See here for some example code:

https://gist.github.com/jaloren/98db79fddf16f844e504

Joe 

[Andrew Palmer]

Hi Bryan, Joseph,

Thank you for your response. Bryan, I have looked into robot lint, but for our requirements it needs to be more customized that we can just using that tool. I did some work with rflint last night, and to me, it seems like aside from the line numbers, it's pretty much identical.

Joseph, I will look into this more, but writing back and forth may get a bit too much overhead when it's running on thousands of files. 

Just to be clear, I'm actually pretty close to accomplishing what I need, just a few small little issues remain. As an example, let's say that I want to check all tags in individual tests in a file, I do as follows:

# Robot test

My_Test_Case

    [Tags]    some tag    another tag

# Code to grab tags

testobj = TestData(source='/path/to/file.robot')

for test in testobj.testcase_table:

    print "Test: %s has tags: %s" %(test.name, test.tags)

# Output is as expected: My_Test_Case: [u'some tag', u'another tag']

Here is the main problem, when I try to do the robot parsing model, I do not get any output aside from the file name. Regardless if I use TestCaseFile or ResourceFile classes, I get nothing. IF I can resolve this, all will be well. Code is as follows:

# Robot resource

My Keyword

    Log    some text

>>> from robot.parsing.model import ResourceFile

>>> recobj = ResourceFile(source='/path/to/resource.robot')

>>> for rec in recobj.keyword_table:

... print "Found Keyword: %s" %rec.name

...

>>>                                     # Note the lack of output

Again, I get same results using TestCaseFile class, am I completely missing something here? I think I've resolved most of the questions I had above, the only thing I'm currently missing is getting the following from a test/resource file:

Library    Something.py

Resource    Resource.robot

Any help with the resource class issue or getting imports would be greatly appreciated. Thanks again Bryan and Joseph for suggestions and comments.

[Bryan Oakley]

I don't understand what you mean by customizable. Rflint was designed specifically to be highly customizable. It was designed to make it easy to write you own rules, and lets you target them to keywords, tests, suites, or the raw files. I don't think it's possible to make it more customizable. Could you explain what you need to do that it won't ket you do?

--bryan

--

[Andrew Palmer]

When I say, customized, it's overall. As an example, we need to be able to check keywords from robot against keywords written in other languages, or parse virtual suites containing robot suites, etc... 

I understand what you're saying, rflint seems to be pretty close to what I'm looking for, and I actually just pieced a few more things together. There is still a few observations. With robot api, I can simply grab a suite object and list out something like settings.force_tags, or test.tags. Maybe I'm missing something in rflint, but from what I see you can get a table, say Settings, in the form of a list such as:

>>> print test.tables[0].statements

[(2-2)[u'Documentation', u'some doc for the test'], (3-3)[u'Suite Teardown', u'Log', u'teardown'], (4-4)[u'Suite Setup Log setup'], (5-5)[u'Force Tags', u'tag1', u'tag2']]

Now from the above, I would still need to:

for line in lines:

    if line[0] == 'Force Tags':

        # Grab the tags from that list.

Same for grabing test doc, tags, teardown, etc...

I'm more just curious at this point as to why in my above example the robot api doesn't seem to work as expected. Also, if I'm missing a simpler way to grab data from the tables in rflint, I apologize and please let me know what I'm missing.

I should have mentioned one of my original concerns, which was that if 100's of people are expected to use this, it's easier to use a package that is already installed, so for rflint, I have to get everyone on board with that, but I guess a temporary solution is easy enough:

try;

    import rflint

expect:

    # install via pip install robotframework-rflint

Thanks again

[Bryan Oakley]

Thanks for the input. I was just curious to see if it's a limitation of rflint, or one of documentation. It looks like it is a little of both. 

Yes, you have to do a tiny bit of traversing of the statements in a suite, testcase or keyword. That's by design because I needed  to catch things that the robot parser would otherwise throw away. With the robot parser, for example, if you had a line like "[Forc tags]" then robot would log  a warning and then throw that statement away. That line would simply not be available anywhere in the parsed data structure. 

If rflint doesn't suit your needs, no harm done. There's certainly something to be said for using the actual robot parser. For a syntax checker I found it to be a bit too difficult to use because you couldn't report errors by line number, and certain types of errors would never show up in the parsed data. If you're mostly concerned with reporting about what's in a well formed suite or resource file, the robot parsers are a pretty good tool. If you want to detect errors that might cause the robot parser to fail or throw warnings, that's where rflint comes in.

I developed rflint when I was on a team that had a bunch of non-technical test-writers. My team was getting hammered by support requests for some of the simplest problems. rflint let us provide them with a tool that could recognize many common mistakes _before_ coming to my team for support, as well as make sure that their tests adhered to our coding conventions.

--bryan

--

[Andrew Palmer]

Hey Bryan,

Yeah one thing I would say, is that more documentation would be helpful, examples too. That's one area where robot api falls short, it's virtually non existent. I've gotten pretty used to using dir() during initial testing so it's not too much of a problem. 

After a bit more frustration with the robot ResourceFile not working, I'm definitely going to be using rflint for some keyword checks. If you have any idea or know someone who does as to why the resource object doesn't seem to work properly in robot api, I would love to know, or if I'm just not doing something correct. 

Your example does give me some idea for use in test files, as we have a header check which checks for valid robot headings. Rflint might offer a simpler approach than my current method.

Appreciate the help again,

Andy

[Tatu Aalto]

Ugh

From this:

>>> from robot.parsing.model import ResourceFile
>>> recobj = ResourceFile(source='/path/to/resource.robot')

You are missing the populate():
>>> from robot.parsing.model import ResourceFile
>>> recobj = ResourceFile(source='/path/to/resource.robot').populate()

More examples, how to use the Robot API for parsing, can be found from here: https://github.com/aaltat/robotframework-from-sublime/blob/master/dataparser/data_parser/data_parser.py

-Tatu

--

[Andrew Palmer]

Tatu, can't thank you enough, I figured it was just something dumb I was overlooking... ugh indeed. Wrongly assumed the same as the api.

Really appreciate your input, just saved me a ton of frustration and pretty much got everything working.

Andy

[Tatu Aalto]

Ugh

Ugh is not meant to be express what it means in English. Ugh is actually greeting from Finnish Donald Duck :-)

But glad that you did get it sorted out.

-Tatu

--

[Andrew Palmer]

Got it, thanks! Finished adding everything in and it is now running in about 1/50th of the time it did while using my own parsing model.

Thanks for all the help/suggestions/comments.

Andy