Selenium WebDriver Is Freezing ?

[Tesfay K. Aregay]

Hi,

I am using selenium webdriver (Firefox driver) to download web pages. It worked fine and downloaded 100000 pages, but then it started to take long time to load pages, and some times make my computer very slow and freeze. I tried disabling cache when creating a new instance of FirefoxDriver but  that didn't help, below is the profile I set.

                profile = new FirefoxProfile();

profile.setPreference("javascript.enabled", true);

profile.setPreference("permissions.default.stylesheet", 2);

profile.setPreference("permissions.default.image", 2);

profile.setPreference("browser.cache.disk.enable", false);

profile.setPreference("browser.cache.memory.enable", false);

profile.setPreference("browser.cache.offline.enable", false);

profile.setPreference("network.http.use-cache", false);

I have set the page load timeout to 3 minutes, the driver is restarted if a page cannot be loaded in 3 minutes, I am using the code below. I don't know were it's going wrong, so I would appreciate a help here.

        

              .......

               for( Dataset dataset : datasetList)

{

long startTime = System.currentTimeMillis();

try

{

if(driver == null)

{

driver = new FirefoxDriver(profile);

}

driver.manage().timeouts().pageLoadTimeout(180, TimeUnit.SECONDS);

driver.get(dataset.getRankedPageUrl());

pageSrouce = driver.getPageSource();

}

catch(Exception ex)

{

datasetDAO.insertError(dataset);

log.info("Load TimeOut : " + dataset.getRankedPageUrl());

driver.quit();

driver = null ;

continue;

}

//create empty file to store the pageSource

File file = createEmptyFile(dataset,pageSource);

                    

               //log the amount of time taken to download the page

               long endTime = System.currentTimeMillis();

               long totalTime = endTime-startTime;

              log.info(" FileName stored in database table.");

              log.info("< Time taken :" + totalTime + " >");

}

               ......