AI LLM ocr hallucinations

[विश्वासो वासुकिजः (Vishvas Vasuki)]

yielded (with formatting and correction suggestions - and a strange error देवासुर-मनुष्यैरपि) -

[[P64]]
किं च हे चिक्लीत,[^1] देवीं-द्योतन-शीलां; मातरम्-अखिल-लोक-जननीम् असज्जननीम्; श्रियं-सर्व-समञ्जस-धारिणीं, मम कुले असत्-पुत्र-पौत्रादिके निवासय च, असत्-कुल-समृद्धिं च त्वम् एव संवर्धय इति अर्थः॥१२॥

देवासुर-मनुष्यैरपि [[अर्च्यन्त्यां|अर्चितां]] पद्म-वासिनीम्।  
त्रयोदश्यां पुनर्देवीं लक्ष्मीं प्रार्थयते हरेः॥

> आर्द्रां पुष्करिणीं पुष्टिं पिङ्गलां पद्म-मालिनीम्।  
> चन्द्रां हिरण्मयीं लक्ष्मीं जातवेदो ममावह॥१३॥

**आर्द्रां**—दयार्द्र-हृदयतया [[तापल्यतप्तान्|ताप-त्रय-तप्तान्]] प्राणिनः दृष्ट्वा दूयमान-चेतस्तया तान् स्व-कटाक्षाञ्चलैः अभिवर्षन्तीम्।

**पुष्करिणीं**—पुष्करवतीम्; श्री-निलय-कर्णिकोदर-निवासिनीम्। “श्री-निलय-धृत-कर-कमल-वर-युताम्” इत्यादि तन्मन्त्रे।

**पुष्टिं**—यज्ञ-क्रिया-सरूपां, परम-याग-भूतां; यष्टव्यां वा।

**पिङ्गलां**—भगवत्-शरणायन-धारार्द्रित-शीतल-मनोहराह्लाद-कर-दिव्य-वर्णाम्।

**पद्म-मालिनीं**—दिव्य-रूपोद्भूत-पद्म-मालालङ्कृताम्; “श्री-निलय-माला-धारिणीम्” इति तन्मन्त्रे।

**चन्द्रां हिरण्मयीम्** इत्यादेः पूर्ववत् योजना। अत्रापि जातवेदः-शब्देन श्रियः पतिम् एव संबोधयति॥१३॥

आदरातिशयेनैव पूर्वोक्त-काम-विशेषतः।  
चतुर्दश्यां पुनः प्राह द्रविणं तत्-सवर्णकम्॥

[^1]: चिक्लीते-ख.

While the formatting is good - maybe it's good to separate the OCR-ing and understanding tasks!

--

--
Vishvas /विश्वासः

[vishal jaiswal]

Good luck with suppressing hallucination 100% while doing accurate ocr. 

--
You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-program...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/CAFY6qgE5uLPHRtAsqofryTBSfozvPK0rik0PTi8-cV8ueruQzg%40mail.gmail.com.

[Shreevatsa R]

LLMs will hallucinate. We've seen this issue to a smaller extent even with pre-LLM OCR that used (very small) language models.

I think the appropriate mitigation is to compare the LLM's OCR output against output from traditional OCR (and do this comparison in an old-school deterministic way with non-smart code, i.e. don't ask an LLM to highlight the differences!). (Maybe you could ask another LLM to judge instances of the comparison, but the comparison itself is best done deterministically.) If using an LLM for proofreading text, the same principle applies: compare against the original text using a regular diff program.

(See things like https://github.com/tylergneill/squinter for example, or just character-level diff tools.)

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/CAKTyUKKiiA9hp4PaT4EG3vJX5EOb0FsyUaBkW71%2BVzKAPAAhSg%40mail.gmail.com.

[Antariksh Bothale]

On the topic of smart diffing, I have found git diff --color-words to produce much more readable diffs if you have made changes to specifically words in a shloka to fix OCR mistakes or such. Instead of showing the whole line as a diff, it specifically colors the word that was changed. 

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/CAKEM%3DPNDiWYpUxdkyCon_JFHpvCN3mhkAVZUJVg4NQfBO9mp8g%40mail.gmail.com.

--

Antariksh Bothale | www.antarikshbothale.com | www.linguistrix.com

[Shreevatsa R]

Yes agreed, word-level diffs are better than line-level diffs, and my point was that character-level diffs are even better. To use the example from the repo linked earlier:

shreevatsa@shreevatsa-personal ~/D/word-diff> cat > f1
khajaṭālaṃ makarandabindusīkaravārṣa puṇdarikamutsaṅge devyā manora-
shreevatsa@shreevatsa-personal ~/D/word-diff> cat > f2
srajaṭālaṃ makarandabindusīkaravārṣi puṇdarikamutsaṅge devyā manoramā-
shreevatsa@shreevatsa-personal ~/D/word-diff> diff f1 f2
1c1
< khajaṭālaṃ makarandabindusīkaravārṣa puṇdarikamutsaṅge devyā manora-
---
> srajaṭālaṃ makarandabindusīkaravārṣi puṇdarikamutsaṅge devyā manoramā-

shreevatsa@shreevatsa-personal ~/D/word-diff [1]> git init
shreevatsa@shreevatsa-personal ~/D/word-diff (main)> git add f1

shreevatsa@shreevatsa-personal ~/D/word-diff (main)> cp f2 f1
overwrite f1? (y/n [n]) y
f2 -> f1

Now compare line-level diffs with word-level diffs with character-level diffs:

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/CAH8J7QaK5gkF7At0BzRfkundzrPx6SiM-Me7LbxWDeAgwoia0A%40mail.gmail.com.

[Karthik]

+1 to character-level diffs

My problem is that I have a terrible experience with Devanagari diffs in the terminal.

I use Kitty which isn't that good with it and the developer isn't too keen, I believe, on fixing it because the fix is not simple.

Any alternative terminals that offer a superior Devanagari experience?

[Antariksh Bothale]

So far it's been universally bad for me with devanagari in terminals

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/c59e8c60-84a6-40c0-b5cf-016deefb77b2n%40googlegroups.com.

Antariksh Bothale | www.antarikshbothale.com | www.linguistrix.com

[Karthik]

Yeah. So I am forced to use a GUI tool. There, you encounter a wrapping issue. Some support it, others don't.

Currently I use a combination of diffuse and meld. Trying out Beyond Compare occasionally.

[Shreevatsa R]

There are no terminals that are good with Devanagari text (or indeed any text with glyphs of varying widths). This is because all terminals are based on a "grid of cells" model (e.g. they assume it is meaningful to talk of (row r, column c) on the screen), which simply doesn't make sense with Indic text. I tried a bunch of them last year and gathered some screenshots here. 

The only shell I've found to work are the ones inside Emacs (M-x shell or M-x eshell). (Coincidentally I just used it today to grep through some files.) It should also be possible to use a Jupyter kernel (which runs in the browser).

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/1c78ca31-08e8-487a-b715-6d1b1196af97n%40googlegroups.com.

[Tyler Neill]

I made Squinter because I wanted character-level diffs with a UX that helped me focus without fatigue, which I didn't find in the 5–6 editors I tried. Turns out I just should have looked further (e.g. Reddit thread); obviously, platform support and continued dev maintenance vary, but there are good alternatives out there. Having found Meld this week, which I really like so far, I think I probably won't use Squinter myself anymore, although it was fun to make and served me well for a few months.