Robot Framework + Selenium Webdriver - get all links on current page and click them all and come back to first page

[jloyzaga]

Hi,

I get that I can use 'Get All Links' and that will return a list but how to loop through that list and click them all and then come back to the original page? 

As I get the link from the list I want to process a few things about each link. Like determine if it is an image link, and when it is actioned does the link point me to another image etc, get the link's properties.

But as yet I don't know how to get the list and then loop through it and even just click them...

Can anyone help me?

Thanks

Joe

[Michał Robaszewski]

Hi
Get all Links works only for link that have ID. So on list you will get lots of empty strings.

Try this :
Crowler
    Selenium2Library.Open Browser    http://www.google.com    ff
    @{hrefs}    Create List
    ${items}    Get Matching Xpath Count    //body//a
    : FOR    ${item}    IN RANGE    ${items}
    \    ${href}    Selenium2Library.Get Element Attribute    //body/descendant::a[${item+1}]@href
    \    log    ${href}
    \    Append To List    ${hrefs}    ${href}
    \    Comment    Do Sth
    Log List    ${hrefs}
    Comment    Do Sth

Take a look on that list. Can contain some javascript:void(0) or '#' . If you operate on list @{hrefs} you can using library Collections clear list from them.

Remove from list ${hrefs} javascript:void(0) #

peace out

[jloyzaga]

got this to work but its not as elegant as yours - opinion? How would you change it and how would you go about making this get the URL for opening in the browser from csv file and then looping through the file until finished?

I then loop through the links and click on them to get screenshots - would it be better to just get the screenshots and click on the links as they are appened to list?

Can't decide - your advice?

Open Browser    http://www.sit.edptest.info/edphome/home.aspx/    firefox
 
 set selenium timeout  10 seconds
 Comment    Count Number Of Linkds on the Page
 ${AllLinksCount}=    Get Matching Xpath Count    //a
 Comment    Log links count
 Log    ${AllLinksCount}
 Comment    Create a list to store link texts
 @{LinkItems}    Create List
 Comment    Loop through all links and store links value that has length more than 1 character
 : FOR    ${INDEX}    IN RANGE    1    ${AllLinksCount}-1
 \    Log    ${INDEX}
 \    ${lintext}=    Get Text    xpath=(//a)[${INDEX}]
 \    ${href}=    Get Element Attribute    xpath=(//a)[${INDEX}]@href
 \    Log    ${lintext}
 \    log to console   ("link text " ${lintext}"href "  ${href} ${INDEX})
 \    ${linklength}    Get Length    ${lintext}
 \    Run Keyword If    ${linklength}>1    Append To List    ${LinkItems}    ${lintext}

[jloyzaga]

Also, how would I then go through that list and click on each href 

On Tuesday, 24 May 2016 19:36:39 UTC+10, Michał Robaszewski wrote:

[jloyzaga]

tried your code and got this

C:\Python27\Scripts\pybot.bat testsuite/

==============================================================================

Testsuite                                                                     

==============================================================================

Testsuite.Example                                                             

==============================================================================

Get All Links                                                         | FAIL |

AttributeError: 'list' object has no attribute 'upper'

------------------------------------------------------------------------------

Testsuite.Example                                                     | FAIL |

1 critical test, 0 passed, 1 failed

1 test total, 0 passed, 1 failed

==============================================================================

On Tuesday, 24 May 2016 19:36:39 UTC+10, Michał Robaszewski wrote:

[Michał Robaszewski]

Easier for you would be grabbing Hrefs from links not text from each //a
When you have hrefs on list you can loop through that list without CSV file.
Its mostly depend what you want achieve. Link are correct or web are looking good .
If you want check links are correct you don't need browser to check that. But if you have to check that site are correct and looks good use go to in for loop.
Clicking that link using //a[@href=${href}] could end with lots off errors cause by browser or long time to loop through list. Not recommended until you have no other choice.

    ${linkitems_length}    Get Length    ${LinkItems}
    @{errors_msg}    Create List
    :FOR    ${INDEX}    IN range    ${linkitems_length}
    \    go to ${your site}
    \    click element ${LinkItems[${INDEX}]}
    \    Capture Page Screenshot

If you have to check that links are good (site is valid and not responding with 400 or 500 error code) use Request library. full example :

Suite Teardown    Run Keyword And Ignore Error    Close All Browsers
Library           Selenium2Library
Library           Collections
Library           requests

Crowler 2
    Open Browser    http://www.google.com    firefox

    set selenium timeout    10 seconds
    Comment    Count Number Of Linkds on the Page
    ${AllLinksCount}=    Get Matching Xpath Count    //a
    Comment    Log links count
    Log    ${AllLinksCount}
    Comment    Create a list to store link texts
    @{LinkItems}    Create List
    Comment    Loop through all links and store links value that has length more than 1 character
    : FOR    ${INDEX}    IN RANGE    1    ${AllLinksCount}-1
    \    Log    ${INDEX}

    \    ${lintext}=    Get Text    xpath=(//a)[${INDEX}]   #<-- for what ? -->
    \    ${href}=    Get Element Attribute    xpath=(//a)[${INDEX}]@href

    \    Log    ${lintext}
    \    log to console    ("link text " ${lintext}"href "    ${href} ${INDEX})

   
    \    ${linklength}    Get Length    ${lintext}  #<-- you are checking text not href ? -->
    \    Run Keyword If    ${linklength}>1    Append To List    ${LinkItems}    ${href}
    Log Many    ${LinkItems}
    Comment Remove Values From List    ${LinkItems}    javascript:void(0)    #<-- don't forget checking content on list -->
    ${linkitems_length}    Get Length    ${LinkItems}
    @{errors_msg}    Create List
    :FOR    ${INDEX}    IN range    ${linkitems_length}
    \    ${ret}    requests.Request    Get    ${LinkItems[${INDEX}]}
    \    ${code}    Run Keyword And Return Status    Should Be Equal As Strings    ${ret.status_code}    200
    \    Run Keyword Unless    ${code}    Append To List    ${errors_msg}    error :${LinkItems[${INDEX}]} | ${ret.status_code}
    ${check}    Run Keyword And Return Status    Lists Should Be Equal    ${errors_msg}    ${EMPTY}
    Run Keyword Unless    ${check}    Fail    Link \ assertion Failed with msg:\n@{errors_msg}

about my code check log.html in attachments. 

C:\GIT\Recommendation\Recommendation\Tests\EndToEndTests\robocze\pybot -L TRACE GroupGoogle.txt
==============================================================================
GroupGoogle                                                                  
==============================================================================
Crowler                                                               | PASS |
------------------------------------------------------------------------------
GroupGoogle                                                           | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  C:\GIT\Recommendation\Recommendation\Tests\EndToEndTests\robocze\output.xml
Log:     C:\GIT\Recommendation\Recommendation\Tests\EndToEndTests\robocze\log.html
Report:  C:\GIT\Recommendation\Recommendation\Tests\EndToEndTests\robocze\report.html

pybot --version :
Robot Framework 3.0 (Python 2.7.11 on win32)

[Michał Robaszewski]

If you need more example or advice, feel free to ask .

After few changes :

*** Settings ***

Suite Teardown    Run Keyword And Ignore Error    Close All Browsers
Library           Selenium2Library
Library           Collections
Library           requests

CrowlerV2
    @{errors_msg}    Create List

    Selenium2Library.Open Browser    http://www.google.com    ff
    @{hrefs}    Create List
    ${items}    Get Matching Xpath Count    //body//a
    : FOR    ${item}    IN RANGE    ${items}
    \    ${href}    Selenium2Library.Get Element Attribute    //body/descendant::a[${item+1}]@href
    \    log    ${href}
    \    Append To List    ${hrefs}    ${href}

    Log List    ${hrefs}
    Remove Values From List    ${hrefs}    javascript:void(0)    \#
    ${linkitems_length}    Get Length    ${hrefs}
    :FOR    ${item}    IN range    ${linkitems_length}
    \    ${flag}    ${ret}    Run Keyword And Ignore Error    requests.Request    Get    ${hrefs[${item}]}
    \    ${code}    Run Keyword And Return Status    Run Keyword If    '${flag}'=='PASS'    Should Be Equal As Strings    ${ret.status_code}
    \    ...    200
    \    Run Keyword Unless    ${code}    Append To List    ${errors_msg}    error :${hrefs[${item}]} | ${ret.status_code}

    ${check}    Run Keyword And Return Status    Lists Should Be Equal    ${errors_msg}    ${EMPTY}
    Run Keyword Unless    ${check}    Fail    Link \ assertion Failed with msg:\n@{errors_msg}

[jloyzaga]

Thanks for that it works great - until it gets a mailto link -  which I want to process differently

but also if the link goes to pdf files, or xml files etc I want to process it differently other than click - how to do that within the code?

I'll attach the log file

also where does it click the href?

[jloyzaga]

This is my code to click the href

Suite Teardown    Run Keyword And Ignore Error    Close All Browsers

*** Settings ***
Library           Selenium2Library
Library           Collections
Library           SauceLabs
Library           requests
*** Test Cases ***
Get All Links
    [Tags]    Links

    Open Browser    http://www.sit.edptest.info/edphome/home.aspx#    firefox

    set selenium timeout    10 seconds
    Comment    Count Number Of Linkds on the Page
    ${AllLinksCount}=    Get Matching Xpath Count    //a
    Comment    Log links count
    Log    ${AllLinksCount}
    Comment    Create a list to store link texts
    @{LinkItems}    Create List
    Comment    Loop through all links and store links value that has length more than 1 character
    : FOR    ${INDEX}    IN RANGE    1    ${AllLinksCount}-1
    \    Log    ${INDEX}
    \    ${lintext}=    Get Text    xpath=(//a)[${INDEX}]   #<-- for what ? -->
    \    ${href}=    Get Element Attribute    xpath=(//a)[${INDEX}]@href
    \    Log    ${lintext}

    \    log to console    "link text " ${lintext}"href "${href} ${INDEX}

    \    ${linklength}    Get Length    ${lintext}  #<-- you are checking text not href ? -->
    \    Run Keyword If    ${linklength}>1    Append To List    ${LinkItems}    ${href}
    Log Many    ${LinkItems}
    Comment     Remove Values From List    ${LinkItems}    javascript:void(0)    #<-- don't forget checking content on list -->
    ${linkitems_length}    Get Length    ${LinkItems}
    @{errors_msg}    Create List
    :FOR    ${INDEX}    IN range    ${linkitems_length}
    \   ${ret}    requests.Request    Get    ${LinkItems[${INDEX}]}

    \   log to console      $(ret)

    \   ${code}    Run Keyword And Return Status    Should Be Equal As Strings    ${ret.status_code}    200

    \   log to console  "gonna link "${LinkItems[${INDEX}]}
    \   click link      ${LinkItems[${INDEX}]}
    \   Capture Page Screenshot
    #\   Selenium2Library.Click Link         link=${LinkItems[${INDEX}]}

    \   Run Keyword Unless    ${code}    Append To List    ${errors_msg}    error :${LinkItems[${INDEX}]} | ${ret.status_code}
    ${check}    Run Keyword And Return Status    Lists Should Be Equal    ${errors_msg}    ${EMPTY}
    Run Keyword Unless    ${check}    Fail    Link \ assertion Failed with msg:\n@{errors_msg}

The first link click gave me this error

ValueError: Element locator 'http://www.sit.edptest.info/edphome/home.aspx' did not match any elements.

[Yashwanth Reddy Pothala]

Hi

i am trying to write a test case on robot frame work using RIDE , Can you please help me in writing the test case for several links like suppose ,if i have multiple links like in the page and  i want to access all the links and verify what is the best way to write the test case.want  to verify all the links in the help with the URL and check if there is no 404 error in the pages.

Thanks,

Yashwanth

[Shiva Prasad Adirala]

Install libraries as mentioned in url : https://github.com/bulkan/robotframework-requests

Import Below Libraries in Test Case:

Library Collections

Library RequestsLibrary

Test Case Should Be Like Below:

Test Case Name

    Create Session    sessionName    http://www.gmail.com  verify=True

    ${resp}=  Get Request  sessionName    /get

    Log    ${resp}

    Should Be Equal As Strings  ${resp.status_code}  404