Dynamically detecting web element attributes without inspecting manually

97 views
Skip to first unread message

Fanik

unread,
May 20, 2022, 4:28:58 PM5/20/22
to Selenium Users

I am trying to scrape from a webpage that changes it's class names and other element attributes dynamically (there is no pattern in the name of the class). I use code in the following format:

element_1 = driver.find_elements(By.XPATH, '//*[contains (@class, "DkEaL")]')

a snippet of the webpage element is:

<button aria-label="cats-over-fence" jsaction="pane.rating.moreCats" jstcache="98" class="DkEaL" jsan="7.DkEaL,0.aria-label,0.jsaction">35 reviews</button>

Is there any way to detect this element as it changes dynamically without manually inspecting it?

Chris Carman

unread,
May 21, 2022, 2:08:22 PM5/21/22
to Selenium Users
No, you need something that's known or at least predictable. If all of the attributes change dynamically in unpredictable ways, you'll have to use something else.

SuperKevy

unread,
May 21, 2022, 4:00:00 PM5/21/22
to Selenium Users
How about finding the button elements as a locater w/ class assuming DKEal. is a consistently name class.  You may need a partial match for class. One sample is not enough to tell,  
Get the button collection and check each objects returned attributes.

element_collection = driver.findElement(By.cssSelector("button[class*=' DkEal. ']"))

Anuv4123

unread,
May 21, 2022, 4:39:03 PM5/21/22
to Selenium-Users
If the button aria-label is unique then you can use that one to find the element even if you have more than one then get the index of that element using driver.findElements



--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7edd834a-0834-41d4-a73f-a150b7d354e4n%40googlegroups.com.

Fanik

unread,
May 22, 2022, 10:45:48 AM5/22/22
to Selenium Users
All the attributes do change dynamically. In that case what should be my approach?

SuperKevy

unread,
May 22, 2022, 1:23:47 PM5/22/22
to Selenium Users
How do I know if there is a backside to the moon since I can't see it.    It's likely that button is within a parent element that can help isolate it.   
Provide more code samples and the solution will become apparent.     An example, if you are looking for the "reviews" button then partial match reviews text within that button isolated within its parent(s) elements (i.e. like div).

David

unread,
May 22, 2022, 8:20:23 PM5/22/22
to Selenium Users
Does what you want to identify have static text that is visually presented to the user? Or does that change along with the dynamic attributes? If the visual text is static, you could attempt to locate by that, crafting an XPath query that involves that text along with other elements along the DOM tree path path.

Fanik

unread,
May 23, 2022, 3:34:04 PM5/23/22
to Selenium Users
The text changes according to the URL, A sample/test of the page would be:

SuperKevy

unread,
May 24, 2022, 1:53:13 PM5/24/22
to Selenium Users
Ok you're scraping data from google maps.   I would suggest reading their terms of service.   
Just what exactly are you testing for?   

Reply all
Reply to author
Forward
0 new messages