Running robot tests against different environments using a selector

[quality goudengids]

Hi,

I'm trying out the robot framework in our test organization. We have different environments (dev, test, stage)

How can I configure my tests so that they remain independent from the different environments but are able to run against them?

I should be able to set a global variable : environment that can be set to dev,tst,stage.

depending on the chosen variable following resources should be taken into account

credentials

database config

Base URL's

Is there a way to accomplish this in a way?

Kind regards,

Geert

[Bryan Oakley]

What we do at my current gig is to put these values in variables rather than hard-coding them into the tests, and then defining these variables in argument files. Then it's just a matter of picking one of the config files when running your tests. For example, to run a test in "qa1" we will do the equivalent of "pybot --argumentfile tests/conf/qa1.args". 

qa1.args looks something like this:

    --argumentfile default.args

    --metadatsa ENVIRONMENT:QA1

    --variable USERNAME: bob

    --variable PASSWORD marley

    --variable DBHOST db.qa1.example.com

    --variable ROOT http://qa1.example.com

We've found that since we only need to define a dozen or so variables, this works fine. Plus, it makes it possible to override what's in the config file if you want (eg: "pybot --argumentfile tests/conf/qa1.args --variable USERNAME:robert". The default.args file means that an environment has to only specify the variables that are unique to that environment.

As a final step, we created our own "pybot" command, so to run the tests in an environment you simply run "ebot qa1" and it loads up the appropriate argument file and passes the appropriate tests to the robot test runner.

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[Tatu Aalto]

Ugh

But if your needs happen to be complex, one can create variables dynamically [1] from variable file [2]. But if your needs are similar to Bryan, I would definitely go with his approach.

-Tatu
Send from my mobile
[1] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#getting-variables-from-a-special-function
[2] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#variable-files

[Azam Alias]

This is what I did in my organisation:

I have a python variable file (environment.py):

def getBaseURL(env):

    if env == 'test':
        domain = 'test.domain.com'

    elif env == 'beta':
        domain = 'beta.domain.com'

    elif env == 'prod':
        domain = 'www.domain.com.'

    return 'http://' + domain

I specify this variable in a global resource file (global_resource.txt):

*** Settings ***
Variables      ../path/to/environment.py


*** Variables ***
${wwwPath}        ${getBaseURL('${env}')}

And import the resource file in each test case / test suite file accordingly:

*** Settings ***
Resource        ../path/to/global_resource.txt

*** Test Cases ***
Test Case 1
    Open Browser  ${wwwPath}

And in pybot script for each environment, I specify which environment to use:

pybot --variable env:prod --exclude testenv-only /path/to/tests/

pybot --variable env:beta /path/to/tests/

For specific inclusion and exclusion of certain test cases / test suites, we use tags at either individual level or suite level, and include & exclude them accordingly at pybot script level.

Test Suite 1:

*** Test Cases ***
    Test Case 1 Not Ready On Prod
        [Tags]  testenv-only
    Test Case 2 To Be Tested On Prod

Test Suite 2:

*** Settings ***
Force Tags  testenv-only

This example is a bit convoluted though. But it works for us so far.

[quality goudengids]

Wow!!!

Thanks guys for the constructive feedback.

I need to look further into it, but I'm happily surprised by the support in this community.

We have a quite complex enterprise architecture here.

We use rest, JMS, different databases, web UIs, mongo etc....

So there'll still some challenges to cope with.

Just managed to send a http post message to one of our REST API's

+1 for selecting this powerful tool to proceed our test automation.

I created an ova file for my testers and developers so that they can start right away with a well configured toolbox.

So everybody has the same baseline. Maybe I might share it sooner or later

Kind regards,

Geert

Op maandag 8 december 2014 15:44:52 UTC+1 schreef quality goudengids:

[quality goudengids]

Ok I managed to import platform specific variables

first of all I created the python file : 

def get_variables(env):


       devparams = {"variableNL":"http://blob.dev.net",
                   "variableFR":"http://blobfr.dev.net",
       }
       testparams = {"variableNL":"http://blob.test.net"}
       stageparams = {"variableNL":"http://blob.stage.net"}
       prodparams = {"variableNL":"http://blob.net" }


 if env == 'dev':
 return devparams
 elif env == 'tst':
 return testparams
 elif env == 'stage':
 return stageparams
 elif env == 'prod':
 return prodparams

Next I defined a global resource file containing : 

*** Settings ***
Variables         environment.py    dev


*** Variables ***

In my tests or other resource files I refer to this global resource file

I created a small sandbox test which logs a variable from the python file : 

*** Settings ***
Resource          ../../GlobalResource.robot


*** Test Cases ***
Use Global variables
    Log    ${variableNL}

 Log says : 

Starting test: Robotframework.Testsuites.Sandbox.Sandbox.Use Global variables
20141211 16:46:18.036 :  INFO : http://blob.dev.net

So this works pretty fine. 

Next I have some concerns though. We have at least more than 40 applications in our enterprise architecture. Each of them has dozens of parameters (DB's, Queue's, URL's)

Will that environment file not grow drastically and hence the readability very worse?

What would be the best approach? split configs in different files? and how do I concatenate the different setting variables into 1 record that I import in robot?

many thanks in advance

[Tatu Aalto]

Ugh

This is question which does not have a single correct answer and therefore it is not easy to answer. But here are some ideas how you can resolve the problem.

The get_variables is just a python function, which is called by Robot Framework. What happens inside of the get_variables  function is totally up to you. You can inherit the variables from other objects and/or classes. The logic for discovering the variables can be as simple or complex as you want and you are not limited in a single file/function/object. Also there are other ways to declare variables [1] in Robot Framework, so do not forget those. Also variable priority and scope is also good [2] to remember

What is best for you, I do not know. It depends how you want use your variables and how you want test your products. Do you test them one by one, in a isolation, do you group then some how or is there some other way you test the applications. I know that I say this often, but do a POC [3], fail fast and try something else. In my humble opinion, by experimenting new things you get the best results and know is this or that good in your case. Happy hunting. 

-Tatu
[1] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#creating-variables
[2] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#variable-priorities-and-scopes
[3] http://en.wikipedia.org/wiki/Proof_of_concept

[Pekka Klärck]

2014-12-11 17:52 GMT+02:00 quality goudengids <quality.g...@gmail.com>:

> Next I have some concerns though. We have at least more than 40 applications
> in our enterprise architecture. Each of them has dozens of parameters (DB's,
> Queue's, URL's)
> Will that environment file not grow drastically and hence the readability
> very worse?
> What would be the best approach? split configs in different files? and how
> do I concatenate the different setting variables into 1 record that I import
> in robot?

As Tatu already commented, there isn't a single best practice that
would work for everyone.

One thing you might want to try is bundling related variables together
to objects. You could have, for example, one class per application and
you could create different instances of it depending on needs. These
objects would then have needed information in attributes that you
could access In Robot data using the extended variable syntax.

If you had, for example, variable file like

def get_variables(env):
if env == 'something':
app = Application('http://10.0.0.42', 'secret', ...)
else:
# ...
return {'app': app, 'other': 'var', 'here', '...'}

class Application(object):
def __init__(self, url, admin_password, etc...,):
self.url = url
self.admin_password = admin_password
# ...

then you could use it in test data like

*** Test Cases ***
Example
Connect to application ${APP.url}
Login as admin ${APP.admin_password}


Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

[quality goudengids]

Hi again,

I would like to come back on this subject since I'm currently still struggling with environment selection on command line

I defined a environment selection python file on the root of my robotframework directory structure

$ROBOTFRAMEWORK_HOME/environment.py contains : 

def get_variables(env):
 devparams = {"param:value-dev"}
 testparams = {"param:value-test"}
 stageparams = {"param:value-stage"}
 prodparams = {"param:value-prod"}

 if env == 'dev':
 return devparams
 elif env == 'tst':
 return testparams
 elif env == 'stage':
 return stageparams
 elif env == 'prod':
 return prodparams

In RIDE I have a resource with variable and argument : 

GlobalResources.robot

*** Settings ***
Variables         environment.py    stage


*** Variables ***

In RIDE I can run environment agnostic tests without any problem.

When I want to run the same tests from command line, I'm not able to pass for example dev to environment.py, so that it picks up the correct base url's for that environment.

pybot --variablefile $ROBOTFRAMEWORK_HOME/environment.py:dev REST.robot runs ok, but when I log a base URL to the test log file, it is still printing the stage URL (stage is set in GlobalResources.robot)

What am I doing wrong with this command? It seems that "dev" is not picked up as argument to select dev parameters in environment.py

Any help or suggestion is welcome

Kind regards,

Geert

[Tatu Aalto]

Ugh

My test environment is growing and I need to take similar approach. I did write small prototype and for me it did work well. My pybot command does look like this:
pybot --variablefile /path/to/variable.py:dev example.robot

I did test in *nix and windows os and the result was same in both os.

-Tatu
Send from my mobile

--

[quality goudengids]

Ok 

I did something similar. Need to check what went wrong though.

Will check if all parameters where defined correctly

Op dinsdag 20 januari 2015 15:01:28 UTC+1 schreef Tatu Aalto:

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-users+unsub...@googlegroups.com.

[quality goudengids]

Ok, Finally I found the time to experiment with command line execution and an environment variable file.

It works like a charm

Integrated a small simple test in Jenkins with the robot framework plugin

Nice integration, I must say

Robot is really fun to work with!

[Kevin O.]

What I have done is some projects is: in a resource file define a variable for the environment, setting a default (and documenting possible values).

*** Variables ***

${MYPROJ_TEST_ENV}    QA1    # 'QA1', 'QA2', or 'STAGE'

And in the resource file that imports the variables file,

Variables         environment.py    ${MYPROJ_TEST_ENV}

This will make a default environment that is easy to override on the command line or RIDE.

python -m robot.run --variable MYPROJ_TEST_ENV:QA2 /path/to/tests

The above would cause get_variables to be called with a value of QA2.

Since a variables file may not be the only item that needs to key off the environment, I think defining a global variable for this is a good idea.

[Bhupesh Dahal]

Hi quality goudengids

Did you figure out what approach worked best for you ? I am in similar situation, trying to design a framework; and want to know if your solution above worked in the long run ?

[Anne kootstra]

We use the same approach as well. In Jenkins we have a matrix build that runs the same test across DEV/TEST/QA and sets a few parameters like browser and environment name in RF via the --v argument. This then allows RF to pick the right values at run time from the (very) large nested YAML file. 

Although I like the simplicity of a single text file, even with the reuse features of YAML, it is difficult to fully oversee it's content. So, in time we may need to convert this to a database and maintain these values there and use the variable pyhton functionality to fetch the relevant info.

Op donderdag 19 februari 2015 20:21:07 UTC+1 schreef Kevin O.:

[Bhupesh Dahal]

Hi Anne, 

Do you have a an example somewhere that i can see?

[Anne kootstra]

Of what part? The YAML file, the Jenkins configuration? 

Op vrijdag 6 april 2018 14:27:48 UTC+2 schreef Bhupesh Dahal:

[Deepesh Som]

Hi Azam,

In my framework I want to call a keyword from variable section directly like you did below i.e.:

*** Variables ***
${wwwPath}        ${getBaseURL('${env}')}

But I am unable to do so, when I am trying this method its throwing an error saying "Variable ${getBaseURL} not found.". So currently I am following a different approach and sending the library class as variable then calling the keyword i.e:

${wwwPath}        ${environment.getBaseURL('${env}')}  # where environment is .py class object

How can I call the keyword directly?

Thanks in advance,

Deepesh