Almost everything you read online about web automation in general is always going to tell you that the recommended best practice for automation testing is to start each test in a new browser so one test does not rely on another. On paper that seems to make since, in practice, not so much.
Yes for most cases having a new browser is best but in same cases it's just not realistic.
Example, so I have been working on automation of this web application using selenium webDriver with TestNG, the test flow is somewhat simple:
- 1. Login
- 2. Select an option from a drop down
- 3. Select search from menu options
- 4. Enter criteria for search
- 5. Click to open one of the results returned
- 6. Perform action XYZ on this page.
Now in step 6 this is where the reason to reuse a browser starts to make sense because their are about 100 or so test cases for just this single page, all of the other test cases have the same steps 1-5, step 6 is the only difference.
Just to sum up the test cases for this page lest just say there are about 25 features on this page and each have the ability to view, edit, remove and add line items to each feature, so yes some test cases fully rely on the other one to work to even move on, I mean no reason to remove a line item if adding it did not work in the first place, and so forth.
The time it takes for the user to complete steps 1-5 is around a full minutes, where step 6 for the most part only takes about 5 seconds on average, so if I was not to share the browser for any of the 100 test would take around 2 hours to run but sharing the browser's the test only takes about 10 min, that's a drastic savings in time, yeah the 2 hour run could be ran in parallel across more than one machine using the selenium grid but that just introduces other potential problems and really you never going to get it down to the 10 minutes it takes to run the other way, you might be able to get it to run in 30 min a best.
Then on top of that there are 20 other sets of this test case and these 20 select different options in step 2 that lead them to another search but have the same about 100 test cases to run on step 6, so this is the part that I would run in parallel with the remote web driver, now the time savings becomes extravagant, even if we were able to get the test run down to 30 min without sharing the browser the total test time for these test cases would be around 10 hours of run time, but sharing the browser's and running the 20 test cases in parallel (assuming you can run only 10 at a time in each case) were only looking at about 20 min to complete all 2000 test cases.
When this is only a small fraction of the total test cases for the application this time saving allows a lot better response time in results, because there are about 5 times this amount of test cases that get ran, when we were not sharing the browser for these test cases it would take 2 days to really get results back on if a build was even good to continue testing, but now we have our answer in about 6 hours, and running these test overnight with saucelabs and Jenkins, let's us know each morning if the build from yesterday is OK to continue with, where before we would have to wait till the following day to have all the test completed, and sometimes we were not able to even run all the test cases on a final build due to the release timeline.
In an upcoming article I will write about the browser part of the selenium framework I developed so help describe the approach I took to setting up test cases to share a browser.