Beautifulsoup all of a sudden does not work
32 views
Skip to first unread message
Harry Allex
May 12, 2021, 8:46:38 PM5/12/21
to beautifulsoup
def lyrics_from_song_path(path):
# gotta go regular html scraping... come on Genius
page_url = "http://genius.com" + path
page = requests.get(page_url)
html = BeautifulSoup(page.text, "html.parser")
# remove script tags that they put in the middle of the lyrics
[h.extract() for h in html('script')]
# at least Genius is nice and has a tag called 'lyrics'!
lyrics = html.find('div', class_='lyrics').get_text() # updated css where the lyrics are based in HTML
return lyrics
# gotta go regular html scraping... come on Genius
page_url = "http://genius.com" + path
page = requests.get(page_url)
html = BeautifulSoup(page.text, "html.parser")
# remove script tags that they put in the middle of the lyrics
[h.extract() for h in html('script')]
# at least Genius is nice and has a tag called 'lyrics'!
lyrics = html.find('div', class_='lyrics').get_text() # updated css where the lyrics are based in HTML
return lyrics
This code used to work a couple days ago, but all of a sudden it stopped working. The html is being extracted correctly, but for some reason html.find is always returning none, even though it would actually get the lyrics a couple days ago. Any idea why this is happening?
Reply all
Reply to author
Forward
0 new messages