Text preservation in the post-industrial age

[विश्वासो वासुकिजः (Vishvas Vasuki)]

The post-oil, post-industrial-age world will arrive (details TM), with much unrest, by 2160 (or 2260 if ice-methane extraction works out). Lack of fertilizers will lead to famine and a big drop in population. Raw materials like plastics, copper, and lithium too will be rarer. Machines - let alone computers - will be scarce. So, electronic data will be ephemeral.(5)

How then to best preserve our knowledge (physical and spiritual) and texts?

Listed some ideas at https://vishvasa.github.io/notes/knowledge-preservation/ .
If you think of other ideas, please let me know.

--

--
Vishvas /विश्वासः

[Arun]

The future is highly uncertain, and we should consider other scenarios as well. For example, most AI experts believe that there's a 50/50 chance of human-level artificial intelligence within the next 100 years. This would be transformative for our scientific and engineering knowledge and also has its own great risks. One paper making the rounds recently has argued that the timelines involved are much shorter and that the geopolital tensions around it could increase enormously very soon.

Otherwise, long-term preservation is the kind of the work the Long Now Foundation has been exploring for some time, and their work on material, locations, etc. for preservation might be worth exploring.

In the very short term, we need to commuciate a basic situational awareness that this era of human life is uncertain and short-lived, and that preservation occurs only with great effort. We might use देश-काल-वस्तु-परिच्छेदः as a framework and think of how to de-risk by देश, काल, and वस्तु.

Arun

[विश्वासो वासुकिजः (Vishvas Vasuki)]

On Sun, 16 Jun 2024 at 22:02, Arun <aru...@gmail.com> wrote:

 

Otherwise, long-term preservation is the kind of the work the Long Now Foundation has been exploring for some time, and their work on material, locations, etc. for preservation might be worth exploring.

This is very useful - thanks! (Many things to check out there, esp their 10k yr optimism)

If someone knows of similar such efforts, please let me know.

The future is highly uncertain, and we should consider other scenarios as well.

Indeed so. Certainties exist though due to thermodynamics, stellar lifecycle, finiteness of resources etc.. - the current rate of growth in energy consumption cannot go on for long; collapse will be voluntary or forced (most likely). 

 

For example, most AI experts believe that there's a 50/50 chance of human-level artificial intelligence within the next 100 years. This would be transformative for our scientific and engineering knowledge and also has its own great risks. One paper making the rounds recently has argued that the timelines involved are much shorter and that the geopolital tensions around it could increase enormously very soon.

It could indeed be that the collapse in energy consumption will be accelerated and forced by AI.

 

In the very short term, we need to commuciate a basic situational awareness that this era of human life is uncertain and short-lived, and that preservation occurs only with great effort.   

Very much agree with this. Very few are aware of the fragility of the industrial age and big-city-life. Particularly, traditional keepers of knowledge should be made to understand the need to return to rural areas. Reminds me of https://vishvasa.github.io/kalpAntaram/vishvAsaH/kriyA-nishchayaH/meta-ritual/presentations/praJNA-dhArAH_saMskRtaM/?printLayout=1&includeStyle= which I presented at a local school. Must polish it and be proactive about presenting such in schools.

 

We might use देश-काल-वस्तु-परिच्छेदः as a framework and think of how to de-risk by देश, काल, and वस्तु.

 

You mean just have strategies specific to/ taking advantage of variations in these parameters?

PS: As an aside, a few friends are considering setting up an agrarian H colony in Latin America (rough logic - It has low population density, plenty land+water. India- dense pop. Famine + war looming post-oil-collapse (150 yrs). Interested folks contact me.

 

Arun

On Friday, June 14, 2024 at 8:21:14 PM UTC-7 Vishvas Vasuki wrote:

The post-oil, post-industrial-age world will arrive (details TM), with much unrest, by 2160 (or 2260 if ice-methane extraction works out). Lack of fertilizers will lead to famine and a big drop in population. Raw materials like plastics, copper, and lithium too will be rarer. Machines - let alone computers - will be scarce. So, electronic data will be ephemeral.(5)

How then to best preserve our knowledge (physical and spiritual) and texts?

Listed some ideas at https://vishvasa.github.io/notes/knowledge-preservation/ .
If you think of other ideas, please let me know.

--

--
Vishvas /विश्वासः

--
You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-program...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sanskrit-programmers/c7d3e862-802d-41f6-9b6d-66f2c5815e20n%40googlegroups.com.

[Arun]

> You mean just have strategies specific to/ taking advantage of variations in these parameters?

Yes. काल and देश are not easy but they are straightforward, i.e replication in a long-term medium across multiple areas with uncorrelated levels of risk (climate, economy, sustainability, geopolitical stability, ...). By वस्तु I mean the form of presentation, since if traditional knowledge isn't presented attractively and accessibly, most people will be either ignorant of it or indifferent to it. Engaging people through an attractive medium is important for growing a supportive community, which in turn further de-risks preservation and increases access to resources and other strategies.

[विश्वासो वासुकिजः (Vishvas Vasuki)]

On Sat, 15 Jun 2024 at 08:50, विश्वासो वासुकिजः (Vishvas Vasuki) <vishvas...@gmail.com> wrote:

The post-oil, post-industrial-age world will arrive (details TM), with much unrest, by 2160 (or 2260 if ice-methane extraction works out). Lack of fertilizers will lead to famine and a big drop in population. Raw materials like plastics, copper, and lithium too will be rarer. Machines - let alone computers - will be scarce. So, electronic data will be ephemeral.(5)

How then to best preserve our knowledge (physical and spiritual) and texts?

Listed some ideas at https://vishvasa.github.io/notes/knowledge-preservation/ .

Page moved to https://vishvasa.github.io/notes/backup/knowledge-preservation/

(https://rebrand.ly/dg-archive is also relevant)

 

Also interesting efforts - 

https://piql.com/case-studies/image-preservation-for-american-institute-of-indian-studies/ 

https://arcticworldarchive.org/service/

Unfortunately their media is good only for a few centuries (maybe).

If you think of other ideas, please let me know.

--

--
Vishvas /विश्वासः

[विश्वासो वासुकिजः (Vishvas Vasuki)]

An article of interest -

https://lil.law.harvard.edu/century-scale-storage/

[विश्वासो वासुकिजः (Vishvas Vasuki)]

Relevant email - 

[INDOLOGY] Projects, websites, universities

Dominik Wujastyk wujastyk at gmail.com
Thu Jul 17 23:31:09 UTC 2025

• Previous message (by thread): [INDOLOGY] Fall Online Language Courses
• Next message (by thread): [INDOLOGY] Projects, websites, universities
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

The recent scare about the Nepal-German descriptive cataloguing data has
yet again shown that universities are unreliable hosts for the preservation
of digital resources.  Long-term members of this forum might remember when
U. Washington shockingly unplugged Blackbox
<https://blackbox.hacc.washington.edu/>, a major repository of Indological
texts, fonts, and software in the 1990s, maintained by Tom Ridgeway. The
Indology website and archive of etexts used to live on a machine at UCL,
but it was closed after I left  that university, without consulting or
warning me.  The University of Cambridge refused to continue hosting John
Smith's Bombay website when he retired.  The Indology website has been able
to take over <https://bombay.indology.info/> the hosting of that incredibly
valuable asset (think: Pabuji, fonts, software,, Mahābhārata, Rāmāyana) .
There are many more examples of websites and archives being just shut down
when faculty members move, or a couple of years after project funding ends.

I don't have a great answer to any of this.  But I do think that the
persistence of digital assets is a vital question for us all and something
we should all think about carefully if we put valued resources on the
internet.

For example, the results of my last funded project are accessible through
http://sushrutaproject.org.  I have tried to push as much important stuff
as I can tot Github and Zenodo (and http://archive-it.org).  But as soon as
I stop paying personally for the registration and hosting at
sushrutaproject.org that gateway website will close down within a few
weeks.  Do I really want to keep paying, out of my pocket, for the rest of
my life?

For the new project that I announced yesterday, I've built the website at
Github.  This costs nothing, so the worry about annual payments is gone.
And as far as I can tell from reading the Github documentation, they do not
delete project repositories, even if they become inactive.  If repositories
are "archived" they simply become read-only (docs
<https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories>).
Github seems very enlightened about long-term preservation.  They save
offline archives at the bottom of a mineshaft in Svalbard - I kid you not.
There are many good features for project work at Github, and it's all
free.  Building a simple one-page website is also extremely quick and
easy.  Building something more complicated, with menus etc., is more
troublesome.  It's significantly harder than using Wordpress (the Microsoft
Word of website creation).  But the up-side is that you get a website that
will persist for years and doesn't require payment.

Within the university world, departments and computing centres are not
going to look after project data and websites in the long term.  The one
institution that actually does think long-term is the university library.
But as far as I can tell, most university libraries are still working out
what their place is in the digital landscape.


Best,
Dominik


--
Dominik Wujastyk, Professor Emeritus, Classical Indian History
University of Alberta

"The University of Alberta is committed to the pursuit of truth,
the advancement of learning, and the dissemination of knowledge
through teaching, research and other scholarly and creative activities and
service."
-- Collective Agreement
<https://www.ualberta.ca/human-resources-health-safety-environment/media-library/my-employment/agreements/2020-2024-collective-agreement---working-version.pdf>
3.01
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://list.indology.info/pipermail/indology/attachments/20250717/4c2e4b89/attachment.htm>

---

• Previous message (by thread): [INDOLOGY] Fall Online Language Courses
• Next message (by thread): [INDOLOGY] Projects, websites, universities
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

More information about the INDOLOGY mailing list

[विश्वासो वासुकिजः (Vishvas Vasuki)]

Sebastian Nehrdich nehrdbsd at gmail.com
Fri Jul 18 01:00:23 UTC 2025

• Previous message (by thread): [INDOLOGY] Projects, websites, universities

• Next message (by thread): [INDOLOGY] Projects, websites, universities
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

Dear Dominik, dear all,

Thank you for raising this vital question about the long-term persistence
of digital infrastructure in our field.

My honest opinion is that university projects, when in doubt, should better
not build platforms*.* The more sustainable and powerful answer is what you
suggest: focus on creating high-quality, open data, and distribute that
freely, not behind the walls of gated databases. Platforms like GitHub (and
private harddrives, websites etc., I get the concerns about only betting on
one horse here) are fantastic for sharing and collaborating on these
datasets, and their lightweight website functions can satisfy the needs of
most academic projects as you point out. It might sound less exciting than
a platform with impressive visualizations, but ultimately, the data is the
asset that will survive institutional changes.

Its easy to get this wrong but software is a service, not a static object.
Without the people who make it work, it's not going to work, and the
academic project-funding system is simply not built to support this
long-term reality. This is why the most successful DH projects are, not by
coincidence, so often centered on key, mission-driven individuals.

Arguing against platforms might sound contradictory coming from someone
primarily known as a platform builder. But that's exactly the point I want
to make: people and software are fundamentally inseparable, and your list
of project examples in the first email demonstrate that very clearly. A
project needs to have a very good answer to the question of long term
development when trying to create digital assets. With data on github or
comparable platforms, that question is easily answered.

Best wishes,

Sebastian

[विश्वासो वासुकिजः (Vishvas Vasuki)]

Claudius Teodorescu claudius.teodorescu at gmail.com
Fri Jul 18 09:23:25 UTC 2025

• Previous message (by thread): [INDOLOGY] Projects, websites, universities
• Next message (by thread): [INDOLOGY] Projects, websites, universities
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

Dear all,

To add just a small technological note to what Dominik rightfully mentioned
above, I would like to point that, even if it is not so famous, GitLab
offers slightly more features than GitLab. Disclaimer: I am not associated
in any way with GitLab, but I am just sharing the conclusions I got after
publishing 16 digital resources (dictionaries, dictionary corpus, GRETIL
search interface, and a search interface for the Indology mailing list).

Besides the more convenient technical features GitLab has, it features a
community edition which can be installed anywhere, by anyone. And this is
pure gold for the research projects, especially after the funding is over.

At the Academy of Mainz, Germany, where I currently work, ALL the digital
resources we produce are stored in Git-type repository, in a GitLab server
that is made available at the land's level, for academic purposes. When I
worked with the Heidelberg University we had a GitLab instance, too, for
the digital resources we developed. This is also currently the case with
the universities or university libraries, respectively, of Bielefeld,
Dresden, Paderborn, and I am sure for others in Germany. I think this is a
model that can be successfully used.

So, my technical advice would be:
1. Use Git-type repositories for research data of any kind: raw research
data (scans, audio files, etc.), processed research data (transcriptions in
TEI format, etc.), and published research data (transcriptions in HTML
format, static indexes for searching in the browser, audio files in
web-friendly formats, images in web-friendly formats, etc.).
2. Separate the raw and processed data to the published data, by storing
the published data only in Pages storage and regenerating it from the
processed data at any change of the processed data. Do not store data in
the publishing format, which is usually HTML.
3. Keep the research data in a repository that is separated to the
repository with the static website that presents the digital resource In
this way, one can have more views of the data, namely: more publishing
formats (HTML, PDF, ePub), linking of the respective digital resource with
other resources, in a corpora, etc. Also, one will not have to process
again the data for every tiny change of the presentation website.

Key concepts to take away: *Git Storage of data*, *static publishing of
data*.

Examples of static digital resources that I published (the search is done
with an in-browser search engine I made and static indexes):
Search interface for the GRETIL corpus
<https://claudius-teodorescu.gitlab.io/gretil-corpus-site/>
Search interface for the Indology mailing list
<https://wujastyk.github.io/INDOLOGY-forum-website/>

Best regards,
Claudius Teodorescu

[विश्वासो वासुकिजः (Vishvas Vasuki)]

(- hindu-vidyA , +claudius)

namaste claudius, 

Search interface for the GRETIL corpus

<https://claudius-teodorescu.gitlab.io/gretil-corpus-site/> is very good.

suppose I wanted a similar search index for - say - https://vishvasa.github.io/AgamaH_brAhmaH/ how would I go about it?

[विश्वासो वासुकिजः (Vishvas Vasuki)]

Thanks for the tip - I tried setting up indexing on a site https://vishvasa.github.io/sanskrit/search/ , and it generates a lot of files - https://github.com/vishvAsa/sanskrit/tree/gh-pages/pagefind/index 

No error in js console. 

Any tips on how to get it to work? (Already tried asking Gemini.)

On Wed, 20 Aug 2025 at 12:48, Claudius Teodorescu <claudius....@gmail.com> wrote:

Hi!

For your data I would recommend to use https://pagefind.app/.

Best regards,

Claudius

--

Cu stimă,

Claudius Teodorescu

[विश्वासो वासुकिजः (Vishvas Vasuki)]

All one needs to do to replicate the problem is to run

git clone --recurse-submodules https://github.com/vishvAsa/sanskrit
cd sanskrit
hugo server

And separately - 

cd sanskrit/build_github 
npx pagefind --site . --output-path pagefind

[Karthik]

@Vishvas

Dump all the texts into an sqlite3 database, create a virtual fts5 table and search to your heart's content.

A slightly more convoluted path, if you want word-level information, is to have three tables:

* document

* word

* document-word (this contains a document-word pair and its index within the document)

[विश्वासो वासुकिजः (Vishvas Vasuki)]

On Thu, 21 Aug 2025 at 14:36, Karthik <kar...@ksml.org> wrote:

@Vishvas

Dump all the texts into an sqlite3 database, create a virtual fts5 table and search to your heart's content.

So you're asking me to write my own indexer, and js search engine :-P . I want something ... less tiresome.

 

--
You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-program...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/e79f1bb7-3b2a-44c5-ac56-cd4e68563014n%40googlegroups.com.

[Karthik]

If you don't want to build a tool, try the algolia.com (the Hacker News search engine) free tier or, if you want a local install, Apache Solr/Lucene.

Whatever you use, a little bit of work is still required.

[Shreevatsa R]

The mentioned Pagefind (website, github) seems like a good solution. What was the problem with it?

In case it helps, here is a list of other such solutions I had assembled a few years ago (2021 it looks like) but never ended up trying any of them:

• Minisearch (blog post, docs, github)
• Lunr.js (website, github, someone's blog post)
• js-search (github, demo)
• flexsearch (github, website) — claims to be the fastest

(For the search page on my Bhartrhari website, I ended up just shipping the entire database to the client and running sqlite in the browser (source), but that's only because I already had the data in a sqlite database, it was pretty small and this was the least effort, etc. I wouldn't recommend it for a static website.)

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/78686974-8774-479a-b048-d973ee249d03n%40googlegroups.com.

[Shreevatsa R]

At your https://vishvasa.github.io/sanskrit/search/ if I open the browser console and manually type:

new PagefindUI({ element: "#search", showSubResults: true })

then it creates a second search box below the existing one, and that one works:

So you may want to just try replacing the "DOMContentLoaded" with "load", or some other thing that makes it run later.

[विश्वासो वासुकिजः (Vishvas Vasuki)]

On Thu, 21 Aug 2025 at 18:26, Shreevatsa R <shree...@gmail.com> wrote:

So you may want to just try replacing the "DOMContentLoaded" with "load", or some other thing that makes it run later.

muito obrigado! I'm so lucky you read this.

 

To view this discussion visit https://groups.google.com/d/msgid/sanskrit-programmers/CAKEM%3DPNd8uBS6sqKWGVu-7SdU%3D91oPLpCsSB2VtDw2kMzvq7ZQ%40mail.gmail.com.