A proposal for the Standard Ebooks community

[Travis Smith]

Hello Standard Ebooks Community,

I've been contributing here in a small way for the last few months — some of you may have come across Gerrata, a tool I built to systematically find errata in Project Gutenberg transcriptions by comparing them against original page scans.

I want to share an automated production pipeline I've been creating over the last 3 months that takes a PG transcription from raw text to a finished, well-typeset EPUB — in 1-2 days per book.  I aimed for 90-95% SE quality...you can judge for yourself at the link below.

The pipeline:
• Builds on SE's work: It begins from SE's tooling and extends it with more automation.  Your work goes from confirming everything with your eyeballs to making sure the pipeline is free of flagged errors.  This kind of work does benefit from coding ability, and isn't suited for everyone.
• It's FOSS: Pipeline code, every book's source, and Gerrata are all on GitHub under CC BY-NC 4.0. Anyone can inspect, fork, or build on it.  Or ignore it. :)

The bigger picture is that there are an estimated 30-40 million public domain works. SE has produced ~1,500 titles. Even at current pace, closing that gap with current methods is a multi-century project, if ever. This is my own answer to the question of how to bridge the gap.

I don't expect this to please everyone, but I'd welcome your thoughts anyway.  I will say up front that what I'm showing you is 'as-is'.  I hope to inspire you, not redesign the software to make it as you would prefer.  This software works quite well for me.

More details: https://impressioneditions.com/about/

And if you're still reading this and are ready to check out the books, my last caveat:  the more recent books on the site have been run without errata correction, at a faster pace.   I was trying to see how fast I could run the pipeline (~5 books per day).  If you want to scrutinize a book for quality, choose one with a Gerrata (green badge) stamp on it.  Those are further down the list a bit here:  
https://impressioneditions.com/ebooks/

Your fellow volunteer in the public domain space,

Travis

[Weijia Cheng]

Thank you for sharing! It is an interesting project. Reading your about page, I think your "gap" framing can be a litte misleading. There are a couple of reasons for why I think this:

1) It's true that SE produces an average rate of 127 books per year, but the current rate is more like 200 or more per year. You can confirm this by looking at the bulk downloads page. This rate has been increasing over time as more and more contributors join the project. So the "gap" between SE and PG is being closed at a faster and faster rate.

2) Only a portion of the PG catalog is relevant to SE. We exclude, for example, foreign-language texts, and outdated instructional books like Glue, Gelatine, Animal Charcoal, Phosphorous, Cements, Pastes and Mucilages (really interesting stuff). We pick a single edition/translation for each SE book, but PG may have multiple editions and translations of the same book. Also in some cases, like short fiction compilations and books that were originally published in multiple volumes, one SE book corresponds to multiple PG books.

3) The same argument applies to the public domain as a whole. How many of the 30-40 million books in the public domain actually fall under the SE collection policy and are interesting enough that a contemporary reader would want to pick it up? If it's something around one percent (not an unreasonable estimate in my opinion), the number comes down to 300,000 or 400,000.

4) There is also something of a "less is more" element with the SE collection. The fact that people work on projects end-to-end means that SE becomes a sort of collectively curated catalog of interesting books in the public domain. I think part of the charm of our project is that our collection is more of a conglomeration of eclectic interests than a top-down effort to curate a canon.

On the practical side, I think it may be difficult for your site to get the traction you would like. My general sense is that this space relies a lot on both first-mover advantage and word of mouth. You may have a hard time getting readers to buy in to what you are doing, especially if part of your sell is that your project is highly automated. Also, as I mentioned in some of our emails off of this list, I think that a more fruitful direction for your transcription framework would be to build a tool that can transcribe a book and convert it into a basic Markdown or HTML format. In my opinion, the work of getting a usable transcription is much more useful and worthy of automation than the PG -> SE reformatting steps.

[David]

Interesting project! I have to say, Weijia's comments (and his recommendation!) make a great deal of sense to me. (I have only a few experiences of transcribing, but it is quite a barrier. Several times I've discovered a book I'd like to produce for SE, only to put that thought aside as there was no "proper" transcription.)

I visited your site, and just a couple observations:
(1) The graphic on the "About" page needs to update "~1400" to "~1500". :)

(2) It seems quite slow (as in, very slow)? Is that a momentary glitch? or a known issue you're already aware of?

Hope this helps! David / Fife, UK

On Thursday, 13 August 2026 at 13:00:58 UTC+1 Weijia wrote:

Thank you for sharing! It is an interesting project. Reading your about page, I think your "gap" framing can be a litte misleading. There are a couple of reasons for why I think this:

1) It's true that SE produces an average rate of 127 books per year, but the current rate is more like 200 or more per year. You can confirm this by looking at the bulk downloads page. This rate has been increasing over time as more and more contributors join the project. So the "gap" between SE and PG is being closed at a faster and faster rate.

2) Only a portion of the PG catalog is relevant to SE. We exclude, for example, foreign-language texts, and outdated instructional books like Glue, Gelatine, Animal Charcoal, Phosphorous, Cements, Pastes and Mucilages (really interesting stuff). We pick a single edition/translation for each SE book, but PG may have multiple editions and translations of the same book. Also in some cases, like short fiction compilations and books that were originally published in multiple volumes, one SE book corresponds to multiple PG books.

3) The same argument applies to the public domain as a whole. How many of the 30-40 million books in the public domain actually fall under the SE collection policy and are interesting enough that a contemporary reader would want to pick it up? If it's something around one percent (not an unreasonable estimate in my opinion), the number comes down to 300,000 or 400,000.

4) There is also something of a "less is more" element with the SE collection. The fact that people work on projects end-to-end means that SE becomes a sort of collectively curated catalog of interesting books in the public domain. I think part of the charm of our project is that our collection is more of a conglomeration of eclectic interests than a top-down effort to curate a canon.

On the practical side, I think it may be difficult for your site to get the traction you would like. My general sense is that this space relies a lot on both first-mover advantage and word of mouth. You may have a hard time getting readers to buy in to what you are doing, especially if part of your sell is that your project is highly automated. Also, as I mentioned in some of our emails off of this list, I think that a more fruitful direction for your transcription framework would be to build a tool that can transcribe a book and convert it into a basic Markdown or HTML format. In my opinion, the work of getting a usable transcription is much more useful and worthy of automation than the PG -> SE reformatting steps.

[Alex Cabal]

Our toolset is advanced enough that, very complex ebooks excluded, the
difficulty is not in *producing* the ebook. An experienced producer
using our toolset could knock out a typical simple prose novel in under
an hour. Once the draft is done, the most time is consumed in reading
the book to proof it, finding cover art, and writing a description. Then
the small details.

I downloaded the advanced epub of Turgenev, the first ebook in the list.
Internally it's all over the place. Endnotes ids are numbered randomly,
filenames are roman numerals, there are vast stretches of invisible
sequential zero width spaces and zero width joiners, no language
semantics, unused CSS files, empty xhtml files, content.opf is not
pretty printed, both descriptions are just the title of the book, the
title SVG is just text (so why is it an SVG at all?), it fails epubcheck...

I think this is a classic case of "an experienced producer could do this
faster by hand, than reviewing and fixing what an AI did".

> More details: https://impressioneditions.com/about/ <https://
> impressioneditions.com/about/>

>
> And if you're still reading this and are ready to check out the books,
> my last caveat:  the more recent books on the site have been run without
> errata correction, at a faster pace.   I was trying to see how fast I
> could run the pipeline (~5 books per day).  If you want to scrutinize a
> book for quality, choose one with a Gerrata (green badge) stamp on it.
> Those are further down the list a bit here:
> https://impressioneditions.com/ebooks/
>
> Your fellow volunteer in the public domain space,
>
> Travis
>

> --
> You received this message because you are subscribed to the Google
> Groups "Standard Ebooks" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to standardebook...@googlegroups.com
> <mailto:standardebook...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/
> standardebooks/9a25ef22-3328-4c66-a143-34eeb4c66150n%40googlegroups.com
> <https://groups.google.com/d/msgid/standardebooks/9a25ef22-3328-4c66-
> a143-34eeb4c66150n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Alex Cabal]

I would also add that this is a good example of why the work we're doing
is important; our editors have chatted about our work and AI privately.

This is presumably a vibe-coded app that uses AI to output vibe-coded
results. IMHO these results are exactly the kinds of epubs that SE has
labored so hard to avoid releasing. Clearly AI has not reached the point
where it can do this work.

Even if it did reach that point at some point in the future, the aspects
of taste in things like compiling editions, selecting cover art, writing
humane and literate descriptions, and the careful attention to detail
that these books require and deserve, means our work will remain
important and vital for a long time to come.

If anything, I'm disappointed that all of the work SE has put in to
championing quality in the ebook building process and its output, has
been interpreted to mean that something like this Turgenev ebook is
acceptable, if we could only release 100 of them per day!

[Travis Smith]

Ok, I guess the gods are not with me this day, but I can try one more time:

The site is loading slow for CDN reasons I can fix soon.  Thanks for calling that out David. 

Alex, try this one instead: https://github.com/Impression-Editions/snorri-sturluson_the-younger-edda/releases/download/v1.0/the-younger-edda.epub

As mentioned, I put the most recent books out under a different assumption to see how quickly I could run the pipeline.  The Gerrata stamped books are more in line with a best-effort approach.  It is still possible that the book doesn't meet the 'standard', to which I must say that I simply have a different one--it's still better than the original material, by quite a lot.

Based on what I've seen of the rate of book creation here, 1 hour for a book is not a claim I can believe without further proof.  I've been seeing 3 days for a short work from an expert, all the way up to 6 months for someone new.  That is totally fine, and I commend these folks.

This was not 'vibe coded', as you imagine it to be, but I am not going to defend myself further on that line.  I present this in good faith, if only to show you that it's possible to work in this way, and it probably won't be the only effort to do so.  My goal is to help close the gap between PG's output and SE's quality level.

And if I somehow upset you, you're in good company--the PG staff didn't appreciate errata reports either. :)

Cooperation or competition, either way, let's make some books!  I'll go grab my crayons.

Travis

You received this message because you are subscribed to a topic in the Google Groups "Standard Ebooks" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/standardebooks/oMPyQCotQh8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to standardebook...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/standardebooks/59db8ac6-d693-4084-9b7d-9d6fe701f670%40standardebooks.org.

[Anthony J. Bentley]

“FOSS” is used inaccurately here—CC-BY-NC is neither free (by the Free Software Foundation’s definition) nor open source (by the Open Source Institute’s definition). The term I would use for this license is “source-available.”

I really appreciate SE’s dedication to keeping its improvements to public domain material within the public domain. It’s a big reason I’m motivated to make small markup improvements and typo fixes to their books as I read them: my contributions are freely usable by everyone in existence, not just SE itself. My suggestion to you would be to consider releasing your books as CC0 rather than the restrictive copyright you’ve embedded these formerly free works in, so that when your software does provide improvements they can cross-pollinate instead of being confined just to the few people willing to use PD material in a locked-down fashion.

[Alex Cabal]

Your first email re. this project some months ago was certainly written
by AI, which is why I didn't read it, and assumed your project was also
AI written.

I am equally unimpressed by your Edda ebook, in which a three-line
dedication is split across three files for some reason, has the same
metadata problems, the same runs of invisible word joiner characters,
the same unused CSS files, is littered with PG-related classes, and also
fails epubcheck. This output is so slipshod that I find it hard to
believe it's not an AI-powered first pass. And if it's not AI, then how
is this supposed to be any better than our own non-AI toolset and workflow?

I'm not upset at you, merely disappointed that all of our work to
champion standards and code quality has been taken to mean that this
kind of output is any any way well made or "95% of SE quality" as you
suggest.

If PG is also rejecting your advances, then perhaps there's some
reflection to be done before continuing.

On 8/13/26 11:11 AM, Travis Smith wrote:
> Ok, I guess the gods are not with me this day, but I can try one more time:
>
> The site is loading slow for CDN reasons I can fix soon.  Thanks for
> calling that out David.
>
> Alex, try this one instead: https://github.com/Impression-Editions/
> snorri-sturluson_the-younger-edda/releases/download/v1.0/the-younger-

> edda.epub <https://github.com/Impression-Editions/snorri-sturluson_the-
> younger-edda/releases/download/v1.0/the-younger-edda.epub>

>
> As mentioned, I put the most recent books out under a different
> assumption to see how quickly I could run the pipeline.  The Gerrata
> stamped books are more in line with a best-effort approach.  It is still
> possible that the book doesn't meet the 'standard', to which I must say
> that I simply have a different one--it's still better than the original
> material, by quite a lot.
>
> Based on what I've seen of the rate of book creation here, 1 hour for a
> book is not a claim I can believe without further proof.  I've been
> seeing 3 days for a short work from an expert, all the way up to 6
> months for someone new.  That is totally fine, and I commend these folks.
>
> This was not 'vibe coded', as you imagine it to be, but I am not going
> to defend myself further on that line.  I present this in good faith, if
> only to show you that it's possible to work in this way, and it probably
> won't be the only effort to do so.  My goal is to help close the gap
> between PG's output and SE's quality level.
>
> And if I somehow upset you, you're in good company--the PG staff didn't
> appreciate errata reports either. :)
>
> Cooperation or competition, either way, let's make some books!  I'll go
> grab my crayons.
>
> Travis
>
>
>
>
>
> On Thu, Aug 13, 2026 at 4:03 PM 'Alex Cabal' via Standard Ebooks
> <standar...@googlegroups.com

> impressioneditions.com/about/> <https://
> >> impressioneditions.com/about/ <http://impressioneditions.com/

> about/>>
> >>
> >> And if you're still reading this and are ready to check out the
> books,
> >> my last caveat:  the more recent books on the site have been run
> >> without errata correction, at a faster pace.   I was trying to
> see how
> >> fast I could run the pipeline (~5 books per day).  If you want to
> >> scrutinize a book for quality, choose one with a Gerrata (green
> badge)
> >> stamp on it. Those are further down the list a bit here:

> >> https://impressioneditions.com/ebooks/ <https://
> impressioneditions.com/ebooks/>

> >>
> >> Your fellow volunteer in the public domain space,
> >>
> >> Travis
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Standard Ebooks" group.
> >> To unsubscribe from this group and stop receiving emails from
> it, send
> >> an email to standardebook...@googlegroups.com

> <mailto:standardebooks%2Bunsu...@googlegroups.com>
> >> <mailto:standardebook...@googlegroups.com
> <mailto:standardebooks%2Bunsu...@googlegroups.com>>.

> >> To view this discussion visit https://groups.google.com/d/msgid/

> <https://groups.google.com/d/msgid/>
> >> standardebooks/9a25ef22-3328-4c66-
> >> a143-34eeb4c66150n%40googlegroups.com
> <http://40googlegroups.com> <https://groups.google.com/d/ <https://
> groups.google.com/d/>
> >> msgid/standardebooks/9a25ef22-3328-4c66-
> >> a143-34eeb4c66150n%40googlegroups.com <http://40googlegroups.com>?

> >> utm_medium=email&utm_source=footer>.
> >
>
> --
> You received this message because you are subscribed to a topic in
> the Google Groups "Standard Ebooks" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/

> topic/standardebooks/oMPyQCotQh8/unsubscribe <https://
> groups.google.com/d/topic/standardebooks/oMPyQCotQh8/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to
> standardebook...@googlegroups.com

> <mailto:standardebooks%2Bunsu...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/

> standardebooks/59db8ac6-
> d693-4084-9b7d-9d6fe701f670%40standardebooks.org <https://
> groups.google.com/d/msgid/standardebooks/59db8ac6-
> d693-4084-9b7d-9d6fe701f670%40standardebooks.org>.
>
> --

> You received this message because you are subscribed to the Google
> Groups "Standard Ebooks" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to standardebook...@googlegroups.com
> <mailto:standardebook...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/
> standardebooks/

> CAOKi2%3DNxHkT7uzRoCtgM%3DcA%3D3MvJVh042YKfg_CV1hd6%3DhhY2Q%40mail.gmail.com <https://groups.google.com/d/msgid/standardebooks/CAOKi2%3DNxHkT7uzRoCtgM%3DcA%3D3MvJVh042YKfg_CV1hd6%3DhhY2Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Vince]

The bigger picture is that there are an estimated 30-40 million public domain works. SE has produced ~1,500 titles. Even at current pace, closing that gap with current methods is a multi-century project, if ever. This is my own answer to the question of how to bridge the gap.

I don’t think the goal of anyone here is to bridge that gap, even if it’s only the 300-400K that Weijia mentioned. The goal is to produce well-crafted PD books. If that takes a couple of days, great, if it takes many months, also great. I’ve never once thought, “If only I could produce 500 books this year!” People who make hand-crafted tables make a conscious choice not to be a part of mass-production, and I think the same is true here.

I want to share an automated production pipeline I've been creating over the last 3 months that takes a PG transcription from raw text to a finished, well-typeset EPUB — in 1-2 days per book.  I aimed for 90-95% SE quality...you can judge for yourself at the link below.

The quality in SE is as much, or more, in the (SE) source than it is in the epub that’s produced from that source. If the source from your process isn’t great quality (and, as noted already, it isn’t), that’s just as bad or worse as if the resulting epub is bad quality.

Based on what I've seen of the rate of book creation here, 1 hour for a book is not a claim I can believe without further proof.  I've been seeing 3 days for a short work from an expert, all the way up to 6 months for someone new.  That is totally fine, and I commend these folks.

Alex doesn’t have to prove anything, he of all people knows better than anyone here what it takes to produce a book. And you apparently missed or ignored the part where he said the hour was producing the book, but then the book has to be proofread (read slower than normal to try to catch typos and other errors), look for cover art, write descriptions, etc. Those things take far more time than the using of the tools to get to that point.

Missing so far from this discussion is the fact that, for me at least, the work of producing the book, i.e. using the tools, is a big part of the joy of doing this work. I wouldn’t automate it if I could.

[Travis Smith]

- Weijia, I thought about the transcription to markdown idea.  Its a good one, but still pretty challenging technically, and not a fight I wish to fight at the moment.

- Licensing is something I'd be happy to re-consider, if there happened to be an honest need for changing it.  I genuinely thought what I chose was quite permissive. Thanks for the comment.

- Alex, yes, I like AI.  Thank you for the life advice.

- Vince, great points, and I get that this is the 'handcrafted table' group.  I can appreciate that, and I do enjoy the outcome.  Its just not my goal.

Since my goal is more books at a higher quality than PG, I'm doing my best not to duplicate your efforts here.  If you want to collaborate to do the same, cool.

[Erin]

Without having read the thread in the greatest detail it does seem that Travis's aims are very different from SE's. So I'm not sure whether this will interest you or not, Travis, but another reason the gap metaphor is inaccurate is that SE hosts work that PG not only lacks but, given its rules and aims, is always going to lack. I am thinking of the omnibus collections in particular. SE has sometimes ended up hosting the first and only complete collection of an author's PD pieces within a particular genre (poetry, plays, short fiction, short works). The content of such collections is the product of extensive bibliographical research, not least because the SE rule is that a collection should contain everything PD that can be found -- all the poems/plays/stories/essays, which the author has usually published in a variety of independent journals, some of them quite obscure and not always available on Internet Archive/HathiTrust. Editors like Weijia have sometimes transcribed pieces from print sources that had to be ordered from/accessed through libraries, because they hadn't been digitised (and perhaps never will be). The Yeats poetry collection produced by Christopher Hapka is a good example of this. The editors can probably remember many other examples.

PG will publish volumes of journals and collections of works, but they will not create collections like these of SE's from an author's individually published pieces, unless those have already been collected and published in an independent anthology that has then become PD itself. This isn't at all a failing of PG given their aims, but it shows that SE's distinct aims lead to valuable work that isn't currently found at PG and never will be unless they change their aims. The rate of ebook release alone tells us nothing about this.

This isn't to repeat Weijia's fourth (good) point about careful curation of a corpus, but is to say further that the SE corpus does contain collections of a kind, and with a value, that can't currently be found at PG, or anywhere else in the public domain, and sometimes not anywhere else at all.

--

You received this message because you are subscribed to the Google Groups "Standard Ebooks" group.

To unsubscribe from this group and stop receiving emails from it, send an email to standardebook...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/standardebooks/625f84f1-5b80-4030-b854-63edaba930a2n%40googlegroups.com.