olibdb.text

[Kurt Pagani]

I wanted to create a sqlite database combining libdb.text and comdb.text. Naively, I converted the files to csv assuming the pointers in the both files would match -- without checking before :(

Apparently the two files are created by dbSplitLibdb() in br-data.boot from a file called olibdb.text which will be deleted when the procedure ends. The function dbGetCommentOrigin() in br-search-boot ought to be capable to assign correct pointers, however, I cannot figure out how the two pointers are  related.

In a nutshell, commenting  --deleteFile '"olibdb.text" in dbSplitLibdb and rebuilding fricas, I got the file olibdb.text that contains all the information I wanted:

https://bitbucket.org/kfp/fricas.db/src/main/

For whatever reasons this file is splitted, then deleted,  it would be beneficial to keep it.

[Waldek Hebisch]

On Mon, Apr 21, 2025 at 02:56:05AM -0700, Kurt Pagani wrote:
> I wanted to create a sqlite database combining libdb.text and comdb.text.
> Naively, I converted the files to csv assuming the pointers in the both
> files would match -- without checking before :(
> Apparently the two files are created by dbSplitLibdb() in br-data.boot from

> a file called *olibdb.text* which will be deleted when the procedure ends.

> The function dbGetCommentOrigin() in br-search-boot ought to be capable to
> assign correct pointers, however, I cannot figure out how the two pointers
> are related.

AFAICS dbGetCommentOrigin is buggy. Correct way to get comment for
libdb line is:

dbComments line

"Pointers" are file offsets: last field in libdb line is file offset
of corresponding entry in comdb. First field of comdb entry is
file offset to corresponding libdb entry. First entry in comdb
is fake, it is used to signal that there is no comment. There
is some support to have multiple comdb entries per libdb entry,
but I am not sure if this is used.

> In a nutshell, commenting --deleteFile '"olibdb.text" in dbSplitLibdb and
> rebuilding fricas, I got the file olibdb.text that contains all the
> information I wanted:
> https://bitbucket.org/kfp/fricas.db/src/main/
>
> For whatever reasons this file is splitted, then deleted, it would be
> beneficial to keep it.

After split this file in not needed by FriCAS. Keeping it would
mostly add confusion. It is not clear to me if splitting info
into libdb and comdb is beneficial. One possible motivation is
using grep for HyperDoc searches. Anyway, I would like to
rework this in not too distant future.

Concerning immediate action, I think we should remove
dbGetCommentOrigin since it is buggy and unused.

--
Waldek Hebisch

[Waldek Hebisch]

On Mon, Apr 21, 2025 at 02:56:05AM -0700, Kurt Pagani wrote:

> I wanted to create a sqlite database combining libdb.text and comdb.text.
> Naively, I converted the files to csv assuming the pointers in the both
> files would match -- without checking before :(

The attached FrICAS file joins back libdb and comdb. Usage like:

)read joindb.input
p1 := "../fr-build110/target/x86_64-linux-gnu/algebra/libdb.text"
p2 := "../fr-build110/target/x86_64-linux-gnu/algebra/comdb.text"
joindb(p1, p2, "sum.out")

--
Waldek Hebisch

[Kurt Pagani]

On 22/04/2025 23:25, Waldek Hebisch wrote:
> On Mon, Apr 21, 2025 at 02:56:05AM -0700, Kurt Pagani wrote:
>> I wanted to create a sqlite database combining libdb.text and comdb.text.
>> Naively, I converted the files to csv assuming the pointers in the both
>> files would match -- without checking before :(
>
> The attached FrICAS file joins back libdb and comdb. Usage like:

Great, you're a wizard. It perfectly worked ('diff' gives nil).

In the meantime I recognized that one can import this file directly,
i.e. without converting to csv first. The sqlitebrowser also accepts the
backtick as a separator. The first character (kind, e.g. cAbelianGroup)
may be easily handled/separated in SQL.

Thank you.