Fanfic archive nearly completed

[Ben Smith]

I figured you guys might like to see this.

www.archive.org/details/fanfictiondotnet_repack

A comprehensive collection of basically every story ever posted to ffnet, complete with a sql database of the metadata collected from the files fanficfare made as it scraped the site.

Organized alphabetically by category. 120gb compressed. About 300 unpacked.
Story Id #1 -10 million.

10-12 million is ongoing.
Enjoy!

[Ranma S]

I have a few questions about this archive mainly revolving around stories that have been deleted from fanfiction.net.

So what would the creation date of this archive be, and exactly how complete is it as both fanfiction.net and some authors themselves have purged stories from the site.

--
Project Home: https://github.com/JimmXinu/FanFicFare/wiki
---
You received this message because you are subscribed to the Google Groups "FanFicFare (was FanFictionDownLoader)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fanfic-downloa...@googlegroups.com.
Visit this group at https://groups.google.com/group/fanfic-downloader.
To view this discussion on the web visit https://groups.google.com/d/msgid/fanfic-downloader/9777dbba-2279-4bab-98b2-f95416a9074c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Smith]

Regarding deleted stories, I ran through all id's from 1 - 10 million, 2 at a time, over the course of a year and a half, starting in 2013. If the story still existed when the script got to it, I have it, at least in some form. I'm not going back over them again. Too much data. But I will keep the current archive going for a while.

[Ranma S]

Thanks, that is exactly what I wanted to know.

On Fri, Apr 15, 2016 at 9:23 PM, Ben Smith <brain...@gmail.com> wrote:

Regarding deleted stories, I ran through all id's from 1 - 10 million, 2 at a time, over the course of a year and a half, starting in 2013. If the story still existed when the script got to it, I have it, at least in some form. I'm not going back over them again. Too much data. But I will keep the current archive going for a while.

--
Project Home: https://github.com/JimmXinu/FanFicFare/wiki
---
You received this message because you are subscribed to the Google Groups "FanFicFare (was FanFictionDownLoader)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fanfic-downloa...@googlegroups.com.
Visit this group at https://groups.google.com/group/fanfic-downloader.

To view this discussion on the web visit https://groups.google.com/d/msgid/fanfic-downloader/e171b86d-f9bc-4279-9e8e-16480e50e7fa%40googlegroups.com.

[savefan...@gmail.com]

Funny this topic should come up now, because right now I'm actually going through that and uploading the individual stories with the metadata to the Internet Archive. I'm managing about ~500 stories an hour. https://archive.org/details/savefanfiction

[Ben Smith]

Good for you, but could you link my archive archive.org/details/fanfictiondotnet_repack

I appreciate the scripting effort you're putting in with the metadata! Thanks!

[Ben Smith]

At that rate, you'll be done in about a year and a half!

[savefan...@gmail.com]

Added the link to your archive in. Thanks very much for repacking and uploading that, it's what inspired me to upload all the stories individually.

And as for the time it'll take, I'm up for it! Luckily my program is split into two parts, a tracker that has a database of the stories that still need to be archived, and many downloaders that get IDs from the tracker, and go archive that story, meaning that I can have many archivers running at once (currently have 20 running from 5 different IPs).

[Ben Smith]

If you're doing it that way, know that there's going to be gaps, some stories were undoubtedly deleted since my archive finished.

I'm running the 10- 12 million range story id's now.

[Jim Miller]

I didn't say anything when you guys were making your announcements, but
I'd appreciate it if you'd take further conversation about it off-list.

Thanks,

Jim

[Jim Miller]

Ben's archive project is outside the scope of this list. I'd

appreciate it if you'd take further conversation about it off-list.

Thanks,
Jim

On 11/23/2016 1:17 AM, Alyssa2cK wrote:
> Dear Ben
>
> Firstly, on behalf of many of the fanfic lovers out there, I must thank
> you for taking the initiative to run the scripts to archive the stories
> available at the time. This sort of backup really can come in handy for
> those of us foolhardy enough to hunt down works since removed. Kudos.
>
> Now- I'm living in something of an internet connection darkage, with
> very poor bandwidth and limited monthly data. Whilst I would love to
> just download the applicable alphabetical zip folder and delve in from
> the https://archive.org/details/fanfictiondotnet_repack, I just want to
> know up until what date exactly you compiled the works?
>
> I.e., did you stop in mid 2015, or so? I'm looking for works in a fandom
> which starts with L (in the Games subsection), and the story title
> starts with S.
>
> I can not for the life of me recall what the FF.net story ID was, and if
> it may be within the 1-10 million range. But the publication date of the
> story ran somewhere between July 2015 and February 2016- which I fear
> falls outside your devoted scouring timeline at that particular
> alphabetical stage.
>
> Anyway, if you could just give me some idea of the exact timelines
> during which the archiving took place, and if you at all think I stand a
> chance to find a fic titled "The Blue room", in the "Life is strange"
> fanfiction archive which was on FF.net between July 2015 and February
> 2016, I would be very much obliged.
>
> Apologies for the novice style questioning.
>
> Regards

>
>
>
>
> On Friday, April 15, 2016 at 9:02:35 AM UTC+2, Ben Smith wrote:
>
> I figured you guys might like to see this.
>
> www.archive.org/details/fanfictiondotnet_repack

> <http://www.archive.org/details/fanfictiondotnet_repack>

[Ripper2509]

Hey, I looked and was unable to find it, but do you have the Destroy All Humans fanfic, "A hazy shade of Winter."

[Jim Miller]

This is off topic for this list.

Discussions here should be about the FanFicFare software.

On Sun, Nov 4, 2018 at 7:12 PM Ripper2509 <overlord...@gmail.com> wrote:

Hey, I looked and was unable to find it, but do you have the Destroy All Humans fanfic, "A hazy shade of Winter."

--

Project Home: https://github.com/JimmXinu/FanFicFare/wiki
---

You received this message because you are subscribed to the Google Groups "FanFicFare -- Downloader for FanFiction" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fanfic-downloa...@googlegroups.com.
Visit this group at https://groups.google.com/group/fanfic-downloader.

To view this discussion on the web visit https://groups.google.com/d/msgid/fanfic-downloader/fbd3a900-a2e2-4266-b5c9-7ded42cad6a0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jim Miller
Retie...@gmail.com

[nilesm...@gmail.com]

How do you find a deleted fanfic? For one of the wow ones titled Xerena Warlock and Sex Goddess Extraordinare it had more chapters than when your scraper got to it. How could I manage to get the rest of the story.

[Jim Miller]

This is a list for the FanFicFare software.

That thread is from ~3 years ago. And it was off-topic then.

If anyone elects to respond, please do so off-list.

Thanks and Happy New Year,

Jim

[Mr M]

Where is the fanfic software?

Thanks,

Mr. M

--
Project Home: https://github.com/JimmXinu/FanFicFare/wiki
--- You received this message because you are subscribed to the Google Groups "FanFicFare -- Downloader for FanFiction" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fanfic-downloader+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/fanfic-downloader.

To view this discussion on the web visit https://groups.google.com/d/msgid/fanfic-downloader/42a93b3a-3599-87d3-405c-68e8b9556929%40gmail.com.

[Jim Miller]

Project Home: https://github.com/JimmXinu/FanFicFare/wiki

[gavn ns]

do you have any easy way I could find a specific fanfic? if you do then this is exactly what I've been looking for.

[Jim Miller]

Please contact Ben off list for any questions about his project.

It is off topic for this list.

Thanks.

On 5/11/2020 3:02 PM, gavn ns wrote:
> do you have any easy way I could find a specific fanfic? if you do then
> this is exactly what I've been looking for.
>
> On Friday, April 15, 2016 at 1:02:35 AM UTC-6, Ben Smith wrote:
>
> I figured you guys might like to see this.
>
> www.archive.org/details/fanfictiondotnet_repack

> <http://www.archive.org/details/fanfictiondotnet_repack>

[rubius]

i checked the inventory for naruto fanfiction and didn't find the one i wanted, is there any other place i can find deadsilence95's naruto fanfiction suicidal & pregnant

[Jim Miller]

This is off-topic for this list. Please find the poster directly or
find another forum.

[Jamie Coady]

hi can i have a link to the archive or any archive. am manely looking for harry potter but any would be good. also what is the link for the fanficfare site or software as it is just turning up an error now. many thanks email is jamieco...@gmail.com

[Jim Miller]

This is reply to something more than 4 years old, which was off topic
then and is still off topic now.

Jim