yahoo stock data

[Betsy]

Hi!

I am new to beautifulsoup but already love it, but I have a problem. I want to get all the stock data available from a yahoo's financial webpage. How do I make sure the table outputs all the data? Right now with my code it only outputs the data available if you do not scroll down. I am sorry if that made no sense, so here is my code. (I left out the import statements)

#"downloads" the source code for the site

sauce = urllib.request.urlopen('https://finance.yahoo.com/quote/PLUG/history?period1=941155200&period2=1609977600&interval=1d&filter=history&frequency=1d&includeAdjustedClose=true').read()

#turns sauce into bs object

soup = bs.BeautifulSoup(sauce,'lxml') #the lxml might not be needed but it helps to be explicit

table = soup.table

table = soup.find('table') # gives the same information

table_rows = table.find_all('tr')

data = []

for tr in table_rows:

    td = tr.find_all('td')

    data.append([i.text for i in td])

    

df = pd.DataFrame(data, columns = ['Date','Open','High','Low','Close','Adj Close', 'Volume'])

    

df.style

Thanks!

[Mauricio Barahona Irizarri]

Hi Betsy,

I solved it once with Selenium.

You open the browser of your choice and scroll down using the webdriver and then download the loaded data. But I guess there's a better way.

Here is the code if you want to take a look: https://github.com/MrExcepcional/kola

Best regards,

Mauricio F. Barahona Irizarri

--
You received this message because you are subscribed to the Google Groups "beautifulsoup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beautifulsou...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beautifulsoup/e4c25047-fd20-4832-aaad-f3546f818dadn%40googlegroups.com.

[FMAPR]

Hi Betsy,

If you have collapsed data, I would also recommend using selenium.
It's as easy as opening the webdriver, finding a selector for the button you want to click that uncollapses the data and you're good to go using bs4.

If you need to click it several times, you know, get crafty!

Regards,

Filipe.

--