Errors in AO3

[pavanag...@gmail.com]

I've started to get some errors when trying to download from AO3. The errors are majorly 

HTTP Error in FFF '525 Server Error:  for url: https://archiveofourown.org/works/39587841/navigate?view_adult=true'(525)

and

HTTPSConnectionPool(host='archiveofourown.org', port=443): Max retries exceeded with url: /works/39734967/chapters/99479802 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001E8EADE32D0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))
https://archiveofourown.org/works/39734967

The links work if I try them manually in the browser, just causing issues in FFN/calibre.

Please advise

[Jim Miller]

The AO3 site is having problems.

See:

https://archiveofourown.org/admin_posts/31204

We estimate the new Elasticsearch servers will be in place by early April. Until then, you might run into the following issues, especially during busy periods:

• all pages loading more slowly
• Elasticsearch-powered pages like search results and work and bookmark listings taking longer to update
• error pages
• automated checks from Cloudflare's Under Attack mode
• stricter rate limiting
• issues with services like the Wayback Machine or Tumblr RSS accounts that rely on bots, scrapers, or other automated tools, which we have deprioritized in favor of traffic from users

Also see:

https://www.mobileread.com/forums/showthread.php?p=4492627&postcount=10407

https://github.com/JimmXinu/FanFicFare/issues/1158

https://github.com/JimmXinu/FanFicFare/issues/1149

https://github.com/JimmXinu/FanFicFare/issues/1163

Some people have had good success using BrowserCache with AO3 with open_pages_in_browser:true

[pavanag...@gmail.com]

Does BrowserCache also work with 'Entire Work' or do we need to go chapter by chapter?

[Jim Miller]

'Entire work' will work, but it's easier to use
open_pages_in_browser:true than manually prepop because FFF adds
view_adult=true to all the URLs.

[pavanag...@gmail.com]

Hi

Can you please confirm what all changes need to be made to personal.ini for this. 

I've currently added the following

[archiveofourown.org]

is_adult:true

username:xxx

password:xxx

use_browser_cache:true

use_flaresolverr_proxy:false

meta_from_last_chapter:true

Should I also add open_pages_in_browser:true and what difference would it make?

[Jim Miller]

I would comment out username and password--FFF can't login via browser cache and it will try for Story URLs from Page.

meta_from_last_chapter is for ffnet only, doesn't affect AO3.

Yes, for AO3, I would also add open_pages_in_browser:true.  When is_adult:true, FFF adds "?view_adult=true" to requests, which you'd otherwise need to add manually to load the cache.

[Pablo]

Hey Jimm one quick question, have you tried to use another domain for Archive of our own? I found this on Reddit, they say that using the https://archiveofourown.gay/ domain stopped giving them the 525 error. I did check and the domain is real and not a joke so maybe we are lucky and they only have cloudfare on their main domain.

[Jim Miller]

Yes.  If you set use_archive_transformativeworks_org:true FFF will still record the usual domain, but will replace it with archive.transformativeworks.org in every request.

This has been working great for me, except it can't for some reason, logging in doesn't work on that version of the site--not even in browser.

However, login does appear to work with archiveofourown.gay, so I'll explore other domain options some more.

[Jim Miller]

I've uploaded a CLI test version that has new option use_archiveofourown_gay

Set use_archiveofourown_gay:true (and don't set use_archive_transformativeworks_org:true) to get FFF to use archiveofourown.gay for all requests.

I've only done minimum testing so far, but it works for me both for stories and login.  I haven't compared to see how up-to-date that version of the site is.

[Pablo]

I just tested a story that published a new chapter a few minutes ago and downloaded it fine. I also updated over a dozen stories I had piled up in my inbox and they all downloaded without any issues, including stories with adult content.

Thank you!

[pavanag...@gmail.com]

I tried this in Calibre's FFF but it didn't seem to take the link.

@Jimm I think you'll have to whitelist it

Also just to confirm,

use_archiveofourown_gay:true 

use_archive_transformativeworks_org:true

will cause FFF to download from .gay domain but the link saved in file will be the standard .org

Will it re-download stories currently downloaded from .org before the cloudfare became an issue or take them as same and just update with newer chapters?

[Jim Miller]

The posted FFF plugin *test* version will accept archiveofourown.gay, but will change it to archiveofourown.org, same as .net and .com.  (I assumed people post here are CLI users and those posting on MR are plugin users.)

You can(should) use the usual archiveofourown.org links.  That's what FFF will remember.

Turning on use_archiveofourown_gay:true will *only* change the domain to .gay for network requests.

If you set *both* use_archive_transformativeworks_org and use_archiveofourown_gay true, you will get only use_archive_transformativeworks_org.  Can't do both.

Existing chapters/stories will not re-download.

[Sian]

Hi All,

I've changed the calibre personal.ini file to show the below (in this order) & it's still not downloading anything. I also tried it with use_archiveofourown_gay:true

[archiveofourown.org]

is_adult:true

slow_down_sleep_time:6

use_view_full_work:true

use_archive_transformativeworks_org:true

use_browser_cache:true

use_flaresolverr_proxy:false

open_pages_in_browser:true

I'm sure it must be something simple I've missed but I can't workout what.

thanks

[Jim Miller]

AO3 is having problems for everyone and has had for months now.

See: https://github.com/JimmXinu/FanFicFare/wiki/ProblematicSites

Use either Browser Cache or a Mirror Site. It is not recommended to try and use both at the same time.  I've had good luck with use_archiveofourown_gay:true lately.

--
Project Home: https://github.com/JimmXinu/FanFicFare/wiki
---
You received this message because you are subscribed to the Google Groups "FanFicFare -- Downloader for FanFiction" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fanfic-downloa...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/fanfic-downloader/43b20e80-0f02-4888-b801-3b6862538ab9n%40googlegroups.com.