log list of libraries imported during the current test run

[Joseph Lorenzini]

Hi all,

Is there anyway to get this info using the public RF api? Note I'd actually like to be able to see all libraries that exist within a particular scope (test case, test suite etc).

Thanks,

Joe 

[Tatu Aalto]

Ugh

I do not remember anyone asking about this kind of question and I do not know where you could obtain such information. Of course this information is somewhere in the robot framework, but perhaps someone from the dev team knows the answer.

But in meanwhile, could you explain why you need this information and what is your goal? Perhaps we can think better ways or workaround to reach your goal.

-Tatu
Send from my mobile

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[Janne Härkönen]

Hi Joe,

On Wed, May 20, 2015 at 8:36 PM, Joseph Lorenzini <jal...@gmail.com> wrote:

Hi all,

Is there anyway to get this info using the public RF api? Note I'd actually like to be able to see all libraries that exist within a particular scope (test case, test suite etc).

The Robot Framework syslog file [1] contains information about execution of suites, creation of suite specific namespaces and libraries and resource files that are taken into use by each suite. There's no run time API for getting this information, though. Is the syslog information enough for your purposes?

hth,

--J

 [1] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#system-log

[Joseph Lorenzini]

Hi Janne/Tatu,

The reason I was looking for this functionality was to have a way to generate a report on library usage across all test cases to guard against hard to debug problems. I want this report so that we can do the following:

1. identify if test cases are using different libraries to do the same thing
2. identify if a test case may use a python module that may not exist on all environments 
3. identify if a library is being used that may have a bug or is undesirable to use for performance or stability reasons.

For example, there are several different SSL and http libraries out there. Alice might write a test case that uses pycurl compiled against openssl. Bob might write a test case that uses HTTP requests (that uses urllib3 with python SSL lib). Due to those implementation differences, the way each test case handles HTTPS could differ -- especially when you get to really new TLS ciphers or god forbid HTTP 2.0.  

This can easily lead to a situation where the test cases are exercising the exact same functionality but one passes while the other fails. 

Following python zen of 'there should only be one way to do it',  I'd like to run a python script that spits out a report that puts libraries into approved and unapproved categories. 

I don't think the robot sys log really fits the bill for the following reasons:

• This is meant as a quick pre-validation check 
• The only way i'd get the information i need is if i ran every single test case -- this could take a really really long time. :)

Joe 

[Bryan Oakley]

The robot API makes it pretty easy to print out a list of imports. For example, this does it for a single file (assuming you don't get parse errors...)

from __future__ import print_function

from robot.parsing import TestData

import sys

def printLibraries(suite, indent=0):

    print(" "*indent + suite.name)

    for lib in suite.setting_table.imports:

        print(" "*indent + "-", lib.name)

    for child_suite in suite.children:

        printLibraries(child_suite, indent=indent+2)

suite = TestData(parent=None, source=sys.argv[1])

printLibraries(suite)

Is that all the information you need? You can also iterate over all of the tests of a test case looking for instances of the "Import Library" keyword. 

[Bryan Oakley]

(oops; meant to say "this does it for a test suite folder and its children, not just a single file)

[Joseph Lorenzini]

Hi Bryan,

Yes, i think that's what  I need. Let me try it out and get back to you.

Thanks,

Joe 

[Tatu Aalto]

Ugh

Bryan's solution to your problem is a great one and most likely I will take it in use also (in one form or other). But it should not be the only solution in your arsenal. In our company, use normal development practice's to minimise these kind of issues for test automation.
* Test automation is done in sprints, including all normal sprint activities.
* There is product owner(s)/architect(s), for test automation, who has final word over test automation (test cases, resource files, libraries, libraries dependencies and so on).
* All test automation is reviewed, a.k. code review. And yes, owner of that part of test automation must participate on all reviews.
* All in-house build libraries have acceptance or/and unit test.
* And I am thinking that pair programming type of activity would serve us also well.

In my mind test automation is coding in so many ways, that many development activities/practices can be applied to it. And like in many development challenges, it is all about communication. Communication, which libraries are available, which dependencies libraries will use. Organizing that all is in logical place, things have logical names and name is based on the domain specific language.  And so on, list is almost endless.

Although this tool wont solve this problem, it's always good reminder: rf-hub [1]. It is great tool to see and search all your keywords and thanks to Bryan from this tool too.

-Tatu
[1] https://github.com/boakley/robotframework-hub

[Bryan Oakley]

Another solution you could employ is robotframework-lint (https://github.com/boakley/robotframework-lint). It is tailor-made for doing these types of checks. In a nutshell, it will parse a whole tree of tests, and then call a series of functions for every suite, testcase and keyword. It's very easy to extend with custom classes to do any types of checks that you want. For example, you could flag the use of certain libraries, or flag when both Selenium2Library and HTTPLibrary are used in the same suite, etc.

[Jari Nurminen]

Hi there,

 

I have been looking for a solution to list the libraries (+ their versions) that are imported and this looks promising...

 

However this doesn't work if you have Library imports inside resource files e.g. something like this:

"suite1.robot" imports library "Collections" and resource file "my_stuff.txt" and that "my_stuff.txt" imports library "String"

The code from Bryan prints only the libs imported directly by the test suites i.e. it would only show "Collections"

 

The code below would also print the libraries imported by the resource files.

NOTE: the below assumes that all resource files are in one directory ("path/to/resource/files/"), because the import_resource() expects a path-to-resource-file as an argument

It would be better load them more dynamically (e.g. search them in PYTHONPATH), but I did't figure out how to do that - If you have a nice solution for that please let me know...

 

from __future__ import print_function
from robot.parsing import TestData

from robot.running.importer import Importer
from robot.parsing.settings import Library, Resource
import sys

 

def printLibraries(suite, indent=0):
    print(" "*indent + suite.name)

 

    for lib in suite.setting_table.imports:
        print(" "*indent + "-", lib.name)

        importer = Importer()
        if isinstance(lib, Resource):
            res=importer.import_resource("path/to/resource/files/"+lib.name)
            for rlib in res.setting_table.imports:
                print(" "*(indent+2) + "-", rlib.name)

 

    for child_suite in suite.children:
        printLibraries(child_suite, indent=indent+2)

 

suite = TestData(parent=None, source=sys.argv[1])
printLibraries(suite)

 

cheerio,

- Jari -

 

------ Originalnachricht ------

Von: "Bryan Oakley" <oak...@bardo.clearlight.com>

An: jal...@gmail.com

Cc: "Janne Härkönen" <janne.h...@iki.fi>; "robotframework-users" <robotframe...@googlegroups.com>

Gesendet: 25.05.2015 17:33:16

Betreff: Re: log list of libraries imported during the current test run

---

This email has been checked for viruses by Avast antivirus software. www.avast.com