Running multiple selenium/py test apps

[bruce]

Hi.

Have a test script that works if I run a single instance. However, If
I run multiple instances as background processes ,the scripts throw
errors.

This should be simple, but it appears something is causing an error.
I'm willing to shell out $$$ for a soln...


sel_test.py 1 --- works


sel_test.py 1 &
sel_test.py 2 &

--- throws errors

running
py 2.6
sel -2.52
centos 6.8



=========================
#test python script
import subprocess
#import re
#import libxml2dom
#import urllib
#import urllib2
import sys, string
import time
import os
import os.path
#from hashlib import sha1
#from libxml2dom import Node
#from libxml2dom import NodeList
#import hashlib
#import pycurl
#import StringIO
#import uuid
#import simplejson
#import webbrowser

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


#----------------------------------------

if __name__ == "__main__":
# main app


# # Opens in safari browser
# browser=webbrowser.get('firefox')
#
# url="http://austin.bncollege.com/webapp/wcs/stores/servlet/TBListView?cm_mmc=RI-_-999-_-1-_-A&catalogId=10001&storeId=65166&langId=-1&termMapping=Y&courseXml=%3C%3Fxml+version%3D%271.0%27+encoding%3D%27UTF-8%27%3F%3E%3Ctextbookorder%3E%3Ccourses%3E%3Ccourse+dept%3D%27ACNT%27+num%3D%271371%27+sect%3D%2708011%27+term%3D%27F16%27+%2F%3E%3C%2Fcourses%3E%3C%2Ftextbookorder%3E"
#
# url="http://austin.bncollege.com/"
#
# out=browser.open_new(url)
#
# print "aa \n"
# print out

# ff="/crawl_tmp/browsetest.dat"
# cmd= "echo '' >"+ff
# proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
# s=proc.communicate()[0].strip()

cmd= "date"
proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
dt=proc.communicate()[0].strip()

ii=sys.argv[1].strip()

driver = webdriver.Firefox()
#driver = webdriver.Chrome()

url="http://austin.bncollege.com/webapp/wcs/stores/servlet/TBListView?cm_mmc=RI-_-999-_-1-_-A&catalogId=10001&storeId=65166&langId=-1&termMapping=Y&courseXml=%3C%3Fxml+version%3D%271.0%27+encoding%3D%27UTF-8%27%3F%3E%3Ctextbookorder%3E%3Ccourses%3E%3Ccourse+dept%3D%27ACNT%27+num%3D%271371%27+sect%3D%2708011%27+term%3D%27F16%27+%2F%3E%3C%2Fcourses%3E%3C%2Ftextbookorder%3E"


out1=''

t=1
while(t):
out=driver.get(url)
#input_='1'

try:
element = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.ID, "createAccountFromCheckout"))
)
out1=driver.page_source

finally:
driver.quit()


if(out1.find('Remarketing')>-1):
t=0
ff="/apps/parseapp2/browsetest.dat"
cmd= "echo '"+ii+" - "+dt+"' >>"+ff
proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
s=proc.communicate()[0].strip()


#time.sleep(2)

out1=out1.encode('ascii', 'ignore').decode('ascii')
print out1

sys.exit()

================================

[David]

Have you tried just running one single script but as background process? I suspect the problem is not running multiple instances but running as background process vs being in foreground.

If that is true, then you could workaround it by just running the multiple processes in the foreground via separate terminals/sessions.

[bruce]

Hey David.

I strongly suspect the issue is a resource allocation issue.

Running the test script once np.

Running the test script in parallel as background for 3-4 instances
seems to work.

Running more, crashes.

The test box, 6G dual core ,centos 6.8

I'm looking to create a parallel distributed parsing process on the
cloud, something like DO where the process would have 2-3 (hopefully)
instances running in parallel. and doing this across a number of
vm/boxes..

> --
> You received this message because you are subscribed to the Google Groups
> "Selenium Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to selenium-user...@googlegroups.com.
> To post to this group, send email to seleniu...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/selenium-users/5809cdac-f6de-438f-96d1-13e799f494fe%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.