Do we need to keep updating to latest?

[Always a student]

Hi All,

I currently have a working solution (with some issues with Firefox that started after updating to latest version). I do see that every few days, NuGet manager shows updates to Webdriver or gecko, chrome or IE driver. I have tried to update in the past and thing have broken due to update, which I had to fix (that took hours and hours of my time evertime I did an  update) to move ahead or just roll back. My question is, do we even need to keep updating even if we have a working solution? I do know sometimes there are new feature but most of the times, I have had broken stuff due to updates. Keeping things like this in mind, is it better to not update unless required (for anything new you are looking for?)

[Jim Evans]

The fact of the matter is that browser vendors continue to update their browsers. You have new versions of Firefox and Chrome every six weeks or so. Your users *will* be using these updated browser versions, so it behooves you to test using these versions. Unfortunately, oftentimes, updating the browser will require an update to the driver for that browser (chromedriver, geckodriver, etc.).

The Selenium project updates more sporadically, but there are usually reasons for it. You can always check the change log for each version (https://github.com/SeleniumHQ/selenium/blob/master/dotnet/CHANGELOG for .NET) to see what changes are in each update.

The short answer is that yes, if you can, you should update. Remember that, at least with NuGet, it’s pretty easy to roll back to a previous version of a package if there’s a bug that gets introduced in a component.

[Always a student]

Makes sense. Thanks Jim

[Sorin Dan]

Hello,

It depends on your automated tests and on what you test. If you test the functionality, I don't think you need to update each time Selenium Web Driver as the functionality happens in back-end. So, as long as you have manual UI test cases, you don't really need to update your test cases.

Best regards,

Sorin

[⇜Krishnan Mahadevan⇝]

Sorin

It's not entirely clear as to what you are recommending.

When you say

>>>>So, as long as you have manual UI test cases, you don't really need to update your test cases.

What do you mean ? If one is having just manual UI tests then there is no selenium involved and so I guess the entire question by the OP itself becomes void no ?

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/5d241ea6-a501-49a6-bbc9-0f69dc1b37d8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Sorin Dan]

Hello Krishnan,

Jim Evans was saying that because the end user has the last version of the browser installed, the tests should be run on the latest browser. I don't agree with that. The changes brought by the browsers don't affect the functionality of the application (web site). So one could have, for example, a vm or a docker machine with an older browser - let's say, for example, FF 45 - and remain to Selenium 2.53 for a while. The only tests which would be important to be made on the latest version of the browser and on different browsers are frond-end tests (UI tests).

If you have a pretty big suite of tests, it may take way too much time to keep it updated for every new version of the browser. And if you work on a continuing developing application you will have to find time also to write new test cases.

The purpose of automated testing is to save time, not to use all the time to maintain the tests. Also, the client should be able to run himself the suite without knowing any programming. How would it look for a company to need to change the product it selled at every couple of months? Because the automated suite is a product you sell and should work also after the contract ends.

Best regards,
Sorin

[Krishnan Mahadevan]

Sorin,

 

Thanks for clarifying what you were hinting at.

 

>>> The changes brought by the browsers don't affect the functionality of the application (web site).

 

This needn’t be true all the time no? Unless and until you look at the changelog/release notes for the latest version of a browser being released, you can’t say that for sure.

From tweaking the way in which the JavaScript engine works, to rendering images differently browsers are free to optimize/fix whatever they find as problems. If you don’t upgrade your test infrastructure, the problem it exposes is that you will be testing on an older version of a browser, but your customers are opening up your website in a newer browser version.

 

Net result: Your customer sees the bug which your tests don’t. That kind of is equivalent to “Surgery was successful but the patient is dead” scenario.

 

But yes, one needn’t be all aggressive in upgrading their versions of either the browsers or selenium for every version that gets upgraded. One can plan this activity such that it’s at a right balance.

Only firefox is going through the churn and that is mostly because the implementation is going through a transition of moving away from a firefox plugin backed support to an independent server binary support. I think once it stabilizes the upgrades should be seamless. Selenium codebase has a very good and elaborate bunch of tests which attempt at vetting out almost everything to the best it can be done.

 

>>> How would it look for a company to need to change the product it selled at every couple of months?

Well, I wouldn’t look at automation suite as a product to start off with. You either provide it as a service, wherein you take ownership of incorporating the changes as and when deemed necessary (or) You resort to a build, operate and transfer model, wherein you hand over the automation suite to your client, and they take ownership of maintaining/enhancing it.  Automation suite at the end of the day is code, and code does require maintenance.

 

>>> Because the automated suite is a product you sell and should work also after the contract ends.

I wouldn’t look at it as a product. Because the actual website that it runs against goes through constant changes (if it’s not changing then it either means that the website is not being used by anyone, or its just a hello world demo sort of a website). So if the website changes, the automated suite has to change. So this analogy doesn’t hold true all the time.

 

 

Thanks & Regards

Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

    --

    You received this message because you are subscribed to the Google Groups "Selenium Users" group.

    To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.

    To post to this group, send email to seleniu...@googlegroups.com.

    To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/20b1027e-2257-4bb6-9174-94fffc77bdea%40googlegroups.com.