how do I run a test suite under CLI with correct order of all test cases?

[Rex Lee]

Hi, 

I've no problem to run all tests which in a test suite from GUI, but when I try to run them under CLI mode, it always fails, I am thinking it might caused by incorrectly cases order, is there any has the same problem?

What I've done are: 

1. Add tests into a test suite name "webtest" and re-order them with correct order. 

2. Run test suite by clicking "Run all tests in suite" from IDE, everything went perfect. 

3. Save this project as selenium.side

3. Under CLI mode, I launch following command:

# selenium-side-runner --filter webtest selenium.side

but it never success. 

in addition, 

The first test is "log into the web", and then real test starts from second test. The first test is okay, but I don't know why it seems like the second test performed without logged in, that is why I thought the runner did not run the tests in sequence, but even I remove the last test (log out web server), the test still fails, so i really don't know why...

Can anyone help?

Regards,

Rex Lee

[Shawn McCarthy]

I am unsure of the fix to your problem. However it is usually good practice to write tests in a way that they can be ran in any order. Then you can run them in parallel to make them faster.

Perhaps you can have 1 UI test that tests the login, then for all the other UI tests, set the cookies needed for login. In my applications I have tested, we have APIs we can call with a username and password, and it will return the cookies needed. From there, you can inject those cookies into the browser.

Even if you have no way to grab the cookies for login without using the UI, it still might be a good idea to have each test login via the UI (so you can run these tests in parallel).

[MEA Engg QA]

A caveat: I am noob, and have been working on Selenium IDE for less than 6 months now - have close to 85 Scripts done, and run them all via Selenium IDE on a daily basis.

I have a similar problem - the Scripts run perfectly when run via the Selenium IDE, but every single Suite fails when run via the CLI using the selenium-side-runner.

I have tried including a single Script in a Suite and running just that one, with the same outcomes - it fails as well.

Every one of my Scripts is independent of others - each Script can be run separately or in a Suite - each Script has a sign in and a sign out so it can be run in any order.

+1 on Rex's question.

Appreciate any guidance from the Gurus here!

Thank you...

MR

----------------