Tool to support the legacy HTML/TSV formats in RF >= 3.2.x

[parlapia...@gmail.com]

Hi,
I'm trying to write a prerun-modifier to support the parsing of HTML/TSV suites while using RF >= 3.2.x.
The aim of the tool is to allow a smooth migration to RF >= 3.2.x for users that have been using HTML and TSV for years, accumulating repos of thousands of suites.
Note: despite tidy, I'm in the unfortunate the case where a big bang migration to .robot will be a serious deterrent for the user.

The idea of the prerun-modifier is to leverage the TestSuiteBuilder of RF 3.1.2 (loaded as separate module with renamed namespaces) to build the legacy suites, and then converts them to the new RF types for populating the tests-tree.
Note: I've used on the version 3.1.2 of RF to also leverage the deprecation warnings thrown for the HTML format, as anyway the direction must be .robot.

The logic of the prerun-modifier is in attachment (I've always worked with Java + RF/Jython, so please ignore any smells of the code).

The prerun-modifier seems to work, however I'm having an issue with the management of resources: given a suite that imports a resource in .html format, I get the error like:
Error in file '...': Invalid resource file extension '.html'. Supported extensions are '.resource', '.robot', '.txt', '.tsv', '.rst' and '.rest'.
That is: it seems that the resource is loaded only after the processing of my prerun-modifier, by the RF 3.2.x runtime, which of course rejects the format.

How could I handle this case? Am I missing something or this is a show-stopper?

Thank you for your help.

Best Regards,
Paolo

[Pekka Klärck]

Hi,

It's true that RF 3.2 and newer don't anymore support HTML or TSV as
data formats and that includes also the built-in Tidy tool. If you
want to convert HTML to plain text format, you need to use RF 3.1.2
and Tidy distributed with it.

This kind of a project to support formats is pretty interesting. The
problem with resource files is that they are parsed only at the
execution time so pre-run modifiers cannot handle them. Well, they
could automatically convert `resource.html` files to `resource.robot`
files and update imports, but it would probably be easier to just
convert those HTML resource files to the `*.robot` format separately.

Cheers,
.peke

ti 8. jouluk. 2020 klo 1.49 parlapia...@gmail.com
(parlapia...@gmail.com) kirjoitti:

> --
> You received this message because you are subscribed to the Google Groups "robotframework-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/robotframework-users/43d52f67-44f8-4187-9b27-22dc155e9f45n%40googlegroups.com.



--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

[Hélio Guilherme]

RIDE can do the conversion for different formats.

I plan to keep this feature in future RIDE versions (at the moment it still uses RF 3.1.2 internally).

--

[parlapia...@gmail.com]

Thank you Pekka and Hélio for your quick replies.

Regarding the resource files, I'll start investigating the proposed approach of automatic conversion + update of references in the referring suites.

In fact, as said previously, in my case the conversion of all the material using tidy 3.1.2 is seen as a last resort.

I will update you about my findings.

Best Regards,

Paolo

[Paolo Parlapiano]

Hi Pekka, Hélio,

please find below my findings on this activity.

As suspected, I couldn't find a way to inject directly to RF 3.2.x the parsed data of resource files in HTML format, even by trying to change the code of RF 3.2.x.

So I've focused on the strategy of converting "on the fly" resource files in the prerun-modifier, i.e. accepting the compromise of creating converted versions of them on file-system.

This strategy seems to be feasible (so far I've made some smoke tests with success), however I had to handle:

1. resource files referencing other resource files
2. path of a resource file defined using variables

To address the point 1. I've extended the Tidy implementation of my local RF 3.1.2 codebase, by adding a procedure that recursively converts a resource tree.

The point 2. is instead a little bit trickier, since in the context of the prerun-modifier I cannot access the system for variables substitution (afaik the Namespace module).

This actually makes sense to me, just because e.g. resource files, potentially affecting the set of available variables, are not processed yet in this phase.

To overcome this, I thought about creating a new Namespace instance (of RF 3.1.2) in my prerun-modifier, which allowed me to eventually handle the variables substitution.

However, I now have at least two open points:

1. I'm not sure that this approach is appropriate / suggested  
   
2. to create the Namespace, I needed to access the settings used to launch the actual Robot Framework runner (3.2.x), but apparently this information is not available
   (as temporary workaround, I've changed the RobotFramework 3.2.x class to the expose the local RobotSettings variable of the main method)

Please find attached the new version of my prerun-modifier, and the amended tidy.py of my local RF 3.1.2 codebase (please focus on the recursive_conv_resource method).

Do you think that this approach could be reasonable?

In the positive case, would it be possible to make the Robot Framework options / settings available to an API client like a prerun-modifier?

Thank you for your help, and merry Christmas to you all :)

Best Regards,

Paolo

To view this discussion on the web visit https://groups.google.com/d/msgid/robotframework-users/a4b98a60-28bb-46e7-a8ee-62fccb014370n%40googlegroups.com.

--

Paolo Parlapiano

[Paolo Parlapiano]

Hi,

did anyone have the chance to take a look at this? 

Thank you 

Regards 

Paolo 

[parlapia...@gmail.com]

Hi,

any chance to take a look at this? 

If the approach described makes sense, I would need to access the parameters used to launch Robot Framework.

Do you think it's feasible, acceptable? Should I create an enhancement request?

Thanks

Paolo

[Hélio Guilherme]

Sorry Paulo,

I have not look at it yet.. Will try to see on next weekend.

To view this discussion on the web visit https://groups.google.com/d/msgid/robotframework-users/d355ae85-e8c9-4bb6-b31c-d7a398a83b3bn%40googlegroups.com.

[parlapia...@gmail.com]

Hi Hélio,

thanks, please let me know if you need further details, or if I can help anyway.

Regards,

Paolo