Using TNTSearch with F3 Jig DB

[Summer White]

I'm posting on here a cross reference to the question I asked on TNTSearch github issues.

I use a php framework called Fat Free Framework and I intend to create an indexer for its own database storage called Jig. It is simple an active cursor implementation which stores data as JSON structures.

I stumbled across TNTSearch but found the documentation is lacking greatly. I've glossed over the code but I suspect it may take a few weeks to fully understand how to work with the code base.

Might anyone give me any hints on how I may tackle this project?

If anyone in this community has suggestions that would be handy dandy :D 

[Anatol Buchholz]

Hey Summer,

I do not know TNT but it looks it´s expected to work with a RDBMS?

(They provide connectors for mysql and postgres and I could not find any

use with NOSQL)

After writing the connector for JIG probably the next question is how to translate

results from:

$data = $db->read('articles');

to results from

$indexer->query('SELECT * FROM articles;');

After this TNT should/could be able to writes it´s index ...

As you might notice I cannot really answer your question. I´m sure

others have better tips. But may I ask you why you´d like to use

it? Is it the fulltext option that ->find and preg_match cannot fullfill?

cheers,

– anatol

[Summer White]

TNT Search has helpers for RDBMS and file systems but it isn't very hard to implement a helper for your own data structure.

In my case I know what sort of data I want to index and how my data is organized. So my implementation will consider how I want to index my own data.

There isn't any particular reason I'd rather use an advanced search feature rather then preg_match. I've written basic regex for searching but this time I'm building a dedicated search engine so I needed something more robust. And because I want to play a bit with TNT Search as it looks quite powerful.

The author responded to my issue with a few examples on how to implement, his response was very snappy. Within an hour. I'm currently implementing some of his examples and learning. I'll post more updates here.

[Anatol Buchholz]

Sounds good!

[ikkez]

I think the effort to get this running doesn't pay off. Jig is a nice, quick storage for a small amount of data. But when you have a jig table (json file) that contains 1000 entries, and you fire multiple search queries at it, it becomes slow with a growing dataset. That's just by the fact that php keeps the whole array in memory and iterates through it with each query. Though I have not tested this yet on php7, which might handle this faster natively. I don't know how TNT Search works and if it does speed up things - great! But I can hardly imagine that it can compensate the Jig slowdown on larger tables. I'd suggest to switch to another db engine when you need to handle bigger data. Jig is fine for editable config storage, like application settings, page tree definitions and such.

[Summer White]

TNTSearch creates a reverse index of the Jig DB. When TNTSearch executes a search it'll do it on a sqlite database and return jig IDs. The benefit of using TNTSearch is that I'll get all the extra features such as fuzzyness and relevancy.

At least thats my logic. I'll wait until hindsight kicks in before commenting on the actual pros and cons.