Thoughts on Claude’s novel

111 views
Skip to first unread message

Tom Gally

unread,
Feb 27, 2025, 7:55:18 PMFeb 27
to hon...@googlegroups.com, swe...@groups.io
I finished reading that Claude-produced novel I posted here last night so that the rest of you don’t have to. Here are my thoughts.

It was surprisingly good. The character depictions, the plot development, and the overall pacing were excellent. The author struck a good balance among the protagonist’s introspection and personal development, his interactions with the other characters, and the plot. I actually wanted to keep reading to the end to find out what happened to the characters—something I was not expecting at all.

There was one major flaw. An entire scene containing a major plot development is repeated at the beginning and the end of the novel, not as some sort of metafictional or structural play but clearly as a mistake. When Claude was doing multiple revisions over several dozens of minutes, I could watch the changes being made in real time in Artifacts, a separate frame that opens within the Claude browser interface. Several times I saw what looked like major blocks of text being added, deleted, or moved. What must have happened is that the scene was in two different locations when I happened to interrupt Claude and it did not notice the problem when I asked it to review the entire novel one more time. If I had either let Claude continue to the end of its reasoning (if it was going to end) or asked for a shorter story that didn’t push the limits of its context window, I’m pretty sure that problem would not have occurred.

The creation and adoption of its own writing style, based on the three samples I gave it, went well. The most obviously borrowing came from Raymond Chandler, which was especially apparent in the opening paragraphs of each of the seven chapters. A couple of examples:

The carpet in the lobby had worn thin along the path from door to elevator, creating a trail as clear as any in the wilderness. The ceiling bore the nicotine memories of a million cigarettes. The elevator wheezed upward with the reluctance of an arthritic forced to climb stairs. Yet for all its decay, the place maintained a vestigial elegance, like an aristocrat forced to pawn the family silver but still dressing for dinner.

March arrived like a debt collector, cold and implacable. The streets remained gray with dirty snow, and the wind that muscled between buildings seemed to have developed a personal vendetta against pedestrians. Michael continued to wear his thin corduroy jacket, financial necessity trumping physical comfort. Each morning, he armored himself in layers of sweaters before making the ten-block trek to Meridian Publishing on West 23rd Street, his breath preceding him in pale ghosts that dissipated in the cutting air.

I’m sure people’s tastes for this type of prose vary, but this is what I asked for. Under that assumption, I think these metaphors and similes are not bad at all.

The characters and plot do echo the outline of that Japanese novel that I had given to it, but not obsessively. For example, the outline I gave Claude contains this:

A [the protagonist] encounters a young woman (M) on his way to work every day. She works at a trading company called Nikka Yoko and always wears her hair in a traditional "ginko-gaeshi" style. A finds himself attracted to M but has never spoken to her. When J visits again, he notices A blushing when they pass M on the street and teases him about having a crush on her.

There’s a similar scene in the Claude novel in which the protagonist becomes interested in a woman he happens to pass each morning on his walk to work in Manhattan. In both novels, a coincidence in the plot development leads the protagonist to learn more about the woman, but the details of that coincidence and how it interacts with the plot as a whole are quite different.

Claude’s descriptions of 1970s New York are okay, but they don’t really bring out the atmosphere as much as I wished. I assume this is because Claude was relying only on its own knowledge from its training data. Seeing how providing a plot outline and style examples helped to enrich those aspects of the novel, I suspect that I could have made the atmosphere more detailed and realistic if I had added to the project knowledge details about 1970s Manhattan, such as articles and page screenshots from The Village Voice, listings from the Talk of the Town section of The New Yorker, etc.—in other words, the sort of materials a human writer of historical fiction uses for inspiration.

Anyhow, based on this one example, it seems that creating readable, enjoyable fiction using an LLM is now viable. I had not expected that.

What’s still missing from this novel and what might be very difficult to produce convincingly is a distinctive individual voice from the writer. No matter how well written, a first-person memoir written by AI is bound to seem phony if the reader knows its provenance.

But for many types of fiction, including not only romance, mysteries, and other pop genres but also middlebrow literary narratives like the one I had Claude write, the author’s individual identity is not, I think, important to most readers. Over the past couple of years, many books have been appearing on Amazon and elsewhere written by AI. The term of art for that kind of writing is “slop.” With proper prompting, it seems that Claude 3.7 Sonnet, at least, is able to rise above the slop level and produce fiction that some people would enjoy reading, especially if they can choose the genres, settings,  and types of characters and stories they are interested in.

If anyone wants to try what I did, using prompts similar to those I posted here yesterday, you could start by having Claude (using the extended thinking function) prepare outlines of novels from the 19th century that are no longer read. Then give Claude those outlines together with ideas for settings, styles, length, etc. as you see fit. I wrote about such forgotten novels a few years ago for the Internet Archive’s blog:

We can expect a new wave of read-but-soon-forgotten novels in the 21st century, too, this time written by AI.

Tom Gally
Yokohama, Japan

Bill Lise

unread,
Feb 27, 2025, 9:59:25 PMFeb 27
to Honyaku E<>J translation list
Tom, 
Wow. The results from this venture into having AI translate a novel (was it self-aware or emulating self-awareness--guess it doesn't matter) should be a wake-up call for translators.
I have no skill at writing or translation fiction, but it looks like the currently progressing demise of commercial, legal, and technical translators could be followed by an existential crisis for fiction translators as well.
That said, compared to the books translated from EN to JA, there is only a tiny number of books that get JA-EN translated, and I suspect that the same tiny group of capable fiction translators that has been handling that smallish demand will be secure for the time being.
That is not the case for other translators, some of whom are in seriously unwarranted AI-denial, others having left, retired, or settled on fixing AI output.
Bill

Tom Gally

unread,
Feb 27, 2025, 10:34:27 PMFeb 27
to hon...@googlegroups.com, swe...@groups.io
Yes, there are many ethical, environmental, and other serious issues
surrounding AI. Tomorrow, I’m giving a talk in Higashi Osaka about the
implications of AI for my other former career, university English
education. I have only ninety minutes to speak, so near the start I
include a slide listing important problems I will not touch on. If
anyone is interested, it’s the fifth slide in my presentation:

https://www.gally.net/temp/20250301jacetkansai/index.html

As with translators and writers, many university educators—and some
students, too—are uncomfortable about AI and do not want to use it.
One my main messages tomorrow will be that that is a valid choice:
conventional teaching methods are still mostly viable, especially for
in-person teaching. However, I will argue that they should be aware of
what the rapidly advancing technology can do, how their students might
be using it, and how it is being used in the world at large.

That was my main purpose in posting my report and comments about the
new Claude. You don’t have to use it or other LLMs if you don’t want
to, and you can encourage others to adopt your position. But hundreds
of millions of people around the world are using the technology now,
strong models are being developed and released not only in the U.S.
but in China, Western Europe, and elsewhere, and attempts to push back
through legal means are making very little progress. This technology
is going to be with us forever, I think, and will affect our lives
more and more in the coming years. I feel I need to understand it as
best I can, and one way to do so is to experiment with it and to
discuss my findings with others.

Google search results, by the way, are classified and ranked using
Google’s proprietary language models, which were almost certainly
trained on the copyrighted texts that Google scooped up through their
Google Books project as well as a huge amount of other copyrighted
material on the open Internet. If you choose not to use AI because of
the intellectual property issues, you probably shouldn’t use Google,
either.

Tom Gally
On the Nozomi between Shin-Yokohama and Osaka

Tom Gally

unread,
Feb 28, 2025, 2:00:40 AMFeb 28
to SWE...@groups.io, hon...@googlegroups.com
Emma Hanashiro wrote:

> The use of AI has been a big point of concern among translators for manga, anime, etc…..

Below is just an anecdote, from someone with nearly zero experience with manga, anime, or the translation thereof. Please take it as that.

First, a preface. A few weeks ago, something I had written about how I use AI when doing translation for my current job was quoted by Simon Willison, a prominent blogger on AI and related topics. Within the next few days, I received emails from three people with comments and requests for advice.

One was developing an app for written conversation between English and Urdu, as he has relatives who speak one or the other language but not both. One nice feature he was working on was adding an LLM-generated commentary to the right of the chat, advising the users about issues in what they wrote that could cause confusion, such as an unspecified gender in English that might be misrendered into Urdu.

Another was a software developer in Mongolia, and he told me how he compared the translation performance of various LLMs for his own personal communication purposes. I'm not sure what languages he used. The English in his email to me seemed written by him, not translated by machine.

The third was an American manga fan. He said he had developed an app to translate not-yet-translated manga into English so that he could read them for his own enjoyment. He said he knew no Japanese at all, but he subscribed to Jump and several other Japanese manga publications in digital form in order to support the artists. He sent me a screenshot of a couple of original Japanese koma and the translations his system—using Google Translate—had produced. He seemed sort of happy with them, but I had to tell him that the translations were absolutely awful.

Many, though not all, of the latest LLMs can accept image input, translate the text in the images into other languages, and interpret what is depicted in the images. So I decided to test recent versions of ChatGPT, Claude, and Gemini with manga. I got five pages of a clean digital copy of a straightforward manga story, and, with Claude’s help, I prepared the following prompt:

Manga Page Analysis Request

I have provided five sequential pages from a Japanese manga, labeled Page 1 through Page 5. Please provide a comprehensive analysis of these pages with the following structure:

Initial Overview

1. First, review all five pages to identify the characters and their relationships
2. Note any recurring visual themes or motifs
3. Confirm your understanding of the reading direction (vertical Japanese text)

Detailed Page Analysis

For each page (1-5), please provide:

Text Analysis

For each text element (in sequential order):
1. Original Japanese text (without furigana)
2. Romanization in Hepburn style (including macrons for long vowels)
3. English translation
4. Story context and significance of this text

Visual Elements

1. Description of any sound effects (onomatopoeia)
2. Description of any visual expression words (mimesis)
3. Explanation of how these elements contribute to the scene

Reading Order

Please analyze the pages in numerical order (1→2→3→4→5) and maintain the natural reading order within each page.

The results were better than what my correspondent had gotten from Google Translate, but they were still terrible. Though the latest LLMs can recognize and convert English text from images nearly perfectly, and horizontal Japanese text fairly well, they all screwed up with the vertical Japanese text in the manga images. Their transcriptions—especially Gemini’s—were full of hallucinations. Most comically, they translated much of the onomatopoeia and mimesis as though they were regular words. I remember in particular ス – – – – used to represent a character standing up straight being translated as “vinegar.”

I told my correspondent that even the best current LLMs didn’t seem good enough to create even a passable translation for personal reading. I wanted to tell him that if he loves manga so much he should try learning Japanese, but that would probably have come across as iyami.

This was several weeks ago. Since then, not only Claude 3.7 Sonnet but also ChatGPT 4.5 have been released (the latter only to people on the highest-paying tier). I ran the same manga and prompt through Claude 3.7 just now. It did a bit better than before, but the transcription still had many misreadings and omissions. Still not usable.

I said at the outset that this was to be “just an anecdote,” but if I may I will attempt to draw a tentative conclusion from this experience: translation can be both supply-driven and demand-driven. Professional translators are on the supply side, and their financial interests align with ensuring the quality of the supplied product. My correspondent, in contrast, is on the demand side, and his interest seems focused on the quantity of the product that he can access. He said he buys a lot of published manga translations, but there’s just so much more that he wants to read that isn’t available. That’s what led him to patch together a system for translating manga with MT and AI, even though the translations are, as I said, terrible.

Tom Gally

James Barker

unread,
Feb 28, 2025, 1:32:44 PMFeb 28
to hon...@googlegroups.com
Tom,

I’ve been a part of this group for years but this is my first time replying to any message. I had a few years in translation/interpreting before transitioning to the technology field where I  have been working as an AI Engineer for several years now.

My job changes almost daily as these new LLM tools are released. I have to say that I agree with your conclusions that these tools are here to stay and that we must learn to use them to our benefit. It’s difficult to admit but several of the hard-won skills I’ve picked up over the past few years (programming for example) can now be more easily utilized by someone with less experience. They likely don’t understand what they’re doing completely, but they can do simple tasks alone.

I now use these tools to research, program, translate, review documentation, get familiar with and learn new subjects, write, and organize thoughts. At my company anyone who doesn’t will likely soon lose their job. 

It’s important for students to learn how to think without these tools, but they must also learn how to use them. At least in the USA, I don’t foresee regulation coming anytime soon. With the release of deepseek, it’s more evident  that at least some of these large models can be downloaded directly to machines in our home and hosted there. We can then work with them or modify them however we like and ,most importantly, cheaply run queries to get results. 

I’ve read that having a self-hosted LLM might be similar in the future to owning a vehicle. As someone who works with technology, this might be where I’m going. 

We must learn how to think critically about the results provided by LLMs. In the same way we learned to discriminate between good sources of information and bad ones on the internet, we must learn how to discriminate likely hallucinations by LLMs and use our hard-won knowledge to verify important details.

If you haven’t tried deep research by Google Gemini, I recommend it.

Joe

----------------------------------
 Joseph    Kane    Blankenship              
-------------------------------------
 Email: jkb...@gmail.com       
-------------------------------------




--
You received this message because you are subscribed to the Google Groups "Honyaku E<>J translation list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to honyaku+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/honyaku/CACnS3NeUYahNszR1v2gXS%3DD5zW6Z83J6%3D%2BK3tP6T7f_%2BKyumng%40mail.gmail.com.

E H

unread,
Feb 28, 2025, 1:33:13 PMFeb 28
to SWE...@groups.io, Daniel Morales via groups.io, Tom Gally via groups.io, hon...@googlegroups.com
Sorry to step in from the anime/manga end.

Nippon Anime & Film Culture Association also released a statement about their concerns about the use of generative AI for translations. 

This is a translated excerpt/summary from Animenomic: 

“Japan should regulate the use of generative artificial intelligence in anime translations in order to protect intellectual property rights, anime labor advocacy group Nippon Anime & Film Culture Association said in a statementpublished last week.

What they’re saying: “Careless AI translations are directly linked to a decline in the reputation of Japanese media content abroad,” NAFCA argues, responding to a public request for comment on the government’s annual plan to promote Japanese IPs.

  • NAFCA contends that translations don’t merely convey meaning and are an integral part of the work being translated, conveying author’s worldview, brand, and reputation.

  • Because AI has the potential to produce incorrect translations, those AI outputs could potentially violate the moral right of the creator to control the integrity of the work.”


    Original Statement: 

The use of AI has been a big point of concern among translators for manga, anime, etc….. 


On Fri, Feb 28, 2025 at 13:29 Ginny Tapley Takemori via groups.io <ginny=tokyotransc...@groups.io> wrote:
Jason

1) I was not the first in this thread to bring up ethics.

2) Way to go, dismissing an entire industry as desperately seeking validation (eye roll).

The Publishers Association



The Society of Authors



ALCS



Authors Guild



(Tom, please note what the ALCA has to say about using copyrighted material for prompts in generative AI, not just for training.)

Discussions are ongoing in much bigger forums than this one, and they are not going away, however much you wish you could close them down.

Ginny



From: SWE...@groups.io <SWE...@groups.io> on behalf of Daniel Morales via groups.io <rupansansei=gmai...@groups.io>
Sent: Friday, February 28, 2025 1:07 PM
To: SWE...@groups.io <SWE...@groups.io>
Cc: Tom Gally via groups.io <tomgally=gmai...@groups.io>; hon...@googlegroups.com <hon...@googlegroups.com>

Subject: Re: [SWET-L] Thoughts on Claude’s novel
 
Jason, 

Please leave the ad hominem attacks off of the list. There's no place for them here.

Daniel

On Fri, Feb 28, 2025 at 12:57 PM Jason Khoh via groups.io <jason=mochiwamo...@groups.io> wrote:
Ginny

Oh but yes, there absolutely is a need to close down these conversations, especially when Tom’s original post was nothing about the ethical considerations of AI

Your actions are akin to the environmental activists who chain themselves to trees: you have to inject your morals and ethics in every conversation, desperately seeking validation from everyone around you

Warmest regards

Jason

Mochiwa Mochiya Pty Ltd

Sent from Smallbiz Yahoo Mail for iPhone

On Friday, February 28, 2025, 12:50, Ginny Tapley Takemori <gi...@tokyotranscultural.com> wrote:

Oh yes, we went through similar legal battles over Google Library too back in the day. I remember it well.

I think there are valid uses for AI that doesn’t infringe copyright and can be a bonus in the classroom and elsewhere (such as assisting with research), and agree that discussions around ethical usage and airing concerns are absolutely vital with or without proposed solutions (no need to close down discussions with dismissive terms like virtue signaling!).

Ginny


From: SWE...@groups.io <SWE...@groups.io> on behalf of Tom Gally via groups.io <tomgally=gmai...@groups.io>
Sent: Friday, February 28, 2025 12:34 PM
To: hon...@googlegroups.com <hon...@googlegroups.com>; swe...@groups.io <swe...@groups.io>
Subject: Re: [SWET-L] Thoughts on Claude’s novel
 
_._,_._,_

Groups.io Links:

You receive all messages sent to this group.

View/Reply Online (#1184) | Reply to Group | Reply to Sender | Mute This Topic | New Topic


Reading this as email? You can view earlier messages online at https://groups.io/g/SWET-L/topics.
Your Subscription | Contact Group Owner | Unsubscribe [emmasatom...@gmail.com]
_._,_._,_

Oroszlany Balazs

unread,
Feb 28, 2025, 5:50:46 PMFeb 28
to hon...@googlegroups.com, SWE...@groups.io
Tom,

This (and the previous standup propt) was eye-opening. Thank you!

Bill,


"That said, compared to the books translated from EN to JA, there is only a tiny number of books that get JA-EN translated, and I suspect that the same tiny group of capable fiction translators that has been handling that smallish demand will be secure for the time being."

Translated Japanese literature is a significant part of all translated fiction, at least here in the UK:

"In 2022, figures from Nielsen BookScan showed that Japanese fiction represented 25% of all translated fiction sales in the UK. The dominance is even more striking this year: figures obtained by the Guardian show that, of the top 40 translated fiction titles for 2024 so far, 43% are Japanese, with Asako Yuzuki’s satirical, socially conscious crime novel Butter topping the list."

https://www.theguardian.com/books/2024/nov/23/japanese-fiction-britain-translation

There is a huge demand for Ja-En literary translators, but I think the real pressure is on light novels and manga - especially as more and more companies see it as a task for "content" manipulation and not an opportunity to attract talent with realistic deadlines and decent pay. There are already Japanese publishers working with more sophisticated automatic manga translation solutions (e.g. https://orange.inc/ ), and they would really like to see their whole catalogue translated automatically. We already saw officially published, AI-translated manga (and it was terrible).

Individual translators may have an opinion on the use of AI in literary translation, but ultimately it is these companies that will make the decisions.

Personally, I will stick with the old-style, hand-crafted translations, both as a reader and as a translator.

Balazs Oroszlany

--
You received this message because you are subscribed to the Google Groups "Honyaku E<>J translation list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to honyaku+u...@googlegroups.com.

Tom Gally

unread,
Feb 28, 2025, 7:16:57 PMFeb 28
to hon...@googlegroups.com
Thanks for posting those comments, Joe. The impact of AI on the software industry seems even bigger than on translation.

If you haven’t tried deep research by Google Gemini, I recommend it. 

At least three products called Deep Research are now available from various companies. Curious how they compare, I ran the same prompt on all three as well as on Claude 3.7 Sonnet with extended reasoning. I chose a topic that was discussed on this list a couple of weeks ago. The results are linked below.

When I have used these deep research products so far, I have found the reports to be useful introductions to subjects I’m not familiar with but want to know more about—much better than a simple Google search. I have been less satisfied with topics I know. I intentionally included video games in the prompt because I do not have any experience translating them.

Be warned that I and others have spotted citations in such reports that are either hallucinated or do not contain the information cited. I have not verified any of the links in the following reports.

I got access to OpenAI’s Deep Research only a couple of days ago, and this is only my second test of it. Some people have reported that it is the best of the lot. I’ll let you all decide (if you care to read the reports).

Prompt

Prepare a report discussing the issue of how the translation of spoken dialogue is handled in novels, plays, subtitled and dubbed films, video games, etc. Focus on how linguistic differences that are present in dialogue in the source language, such as dialect, register, gender, age, etc., are represented or not in the target language.


Tom Gally

Bill Lise

unread,
Feb 28, 2025, 7:31:30 PMFeb 28
to hon...@googlegroups.com
I’m sure translated Japanese literature is quite significant overseas. I had in mind the huge disparity between JA-EN and EN-JA, the latter being much larger, at least as I understand it. 
Additionally, taking a broader view of the total demand, I think it’s pretty safe to say that entertainment-related translation from Japanese to English is only a tiny fraction of the total Japanese-to-English translation demand. It’s very popular and high profile, but not a great volume compared to other types of JA-EN translation.
That said, the high-volume fields are generally being taken over by AI, while entertainment-related translation and literature are probably still safe, for a while anyway. 
Bill

*********************************************
リゼ ウィリアム
Bill Lise
*********************************************


Tom Gally

unread,
Feb 28, 2025, 9:16:59 PMFeb 28
to hon...@googlegroups.com
While the amount of commercially translated and published Japanese fiction is indeed increasing, it is still a tiny fraction of all that is written and available in Japanese. Among light novels alone, a search at Amazon Japan suggests that three or four are published on average each day. A dozen or more monthly and quarterly magazines publish huge amounts of fiction of all genres—to say nothing of the vast number of 同人誌 and other self-supported publications, including online. And no human translator is ever going to translate 『大阪の宿』 into English, either.

In the past, people who don't read Japanese could not access any of that. Now AI offers a way.

Tom Gally
Killing time in a Doutor in Tsuruhashi Station
Reply all
Reply to author
Forward
0 new messages