Capybara with PhantomJS / Poltergeist: Selecting value from drop-down doesn't trigger Ajax call

[trashymctrash]

For non-commercial purposes I am trying to build a scraper. I have worked with PhantomJS before, and it usually works like a charm. However, I have stumbled upon the following problem:

What I want to do:

• Go to http://de.soccerway.com/national/germany/bundesliga/2010-2011/regular-season/r11840/
• Click on the button 'per Spieltag'
• Then select an option from the drop-down menu that holds the matchdays
• Wait for Ajax call to be complete and then get the information I need

Unfortunately, I can select the option I want to select, but nothing happens afterwards. I created a mini script to demonstrate this behavior:

require 'capybara/poltergeist'
require 'capybara/dsl'
include Capybara::DSL

Capybara.register_driver :poltergeist do |app|
  Capybara::Poltergeist::Driver.new(app, :js_errors => false)
end

Capybara.default_driver = :poltergeist
Capybara.javascript_driver = :poltergeist
Capybara.default_wait_time = 20
Capybara.ignore_hidden_elements = true
Capybara.current_session.driver.resize(1200, 1000)

visit('http://de.soccerway.com/national/germany/bundesliga/2010-2011/regular-season/r11840/')

# click on button 'per Spieltag'
find(:xpath, ".//a[text() = 'per Spieltag']").click

# drop down menu with matchdays
matchday_selector = find(:xpath, ".//select[@name='page']")

# save screenshot 'before'
save_screenshot('before.png', full: true)

# for demonstration purposes, select option with value '5'
matchday_selector.find(:xpath, ".//option[@value='5']").select_option

# wait some time for ajax call to be complete and save screenshot 'after'
sleep 5
save_screenshot('after.png', full: true)

If you compare the before and after screenshots, you will see that the number of the drop-down menu has changed, but the content of the table didn't. Also, the dropdown menus are not displayed at the correct position, but instead are shown in the top left corner of the screenshot.

If I use Selenium as my default driver, then it works, but I'd rather user PhantomJS because it is much faster in my experience.

How can I get PhantomJS to trigger the Ajax call?

I use PhantomJS 1.9.1, Capybara 2.1.0 and Poltergeist 1.3.0

[Ronnie Bermejo]

I am facing something similar. Wondering if you were able to spot the problem/fix it.

I cannot click a dropdown submenu. 

Thank you,

-r.

[trashymctrash]

I eventually gave up on it, sorry. :(

[Jobins John]

HI,

DId anyone find any solution for the above mentioned problem?

Please share if anyone has found any solution?

[djber...@gmail.com]

I had issues like this, depending on the way the page is coded I found I had to use a trigger('click') or trigger('modify') so that the appropriate function is called by the web page.

[Jason Wozniak]

I'm having the same problem when trying to scrape Amazon.  When I select an item from a dropdown using seleniums Select(), the item is clearly selected, but some javascript isn't executing.  

It's a real problem because in order to hit the continue button, it must know that an option was selected.  Something somewhere is not getting triggered...

TLDR: any updates?  I'm having the same issue !