Announcement: JonoonDB - A database for flatbuffers

[Zarian Waheed]

I am very happy and excited to announce JonoonDB. An open source multi model database that supports flatbuffers as a native storage format.

 

Website: www.jonoondb.com

 

The database is in alpha stage. There is still some way to go before it is feature complete and production ready. But there is enough functionality in place that it can be useful for many use cases. Some of the features available today are: 

1. Store your flatbuffers data reliably in jonoondb (protected against process and OS crashes).
   
2. Retrieve the flatbuffers data.
   
3. Query the flatbuffers data using SQL. Full SQL support, select, group by, order by, where clause, aggregations etc.
   
4. Index flatbuffers data for even faster query execution.
   
5. Compress the data as you are writing it in the database.
   
6. No need for ORM since the database natively supports flatbuffers.
7. View flatbuffers data inside JonoonDB using jonoondb_cli tool. A  command line tool for interacting with databases. 

Currently JonoonDB is available as a library with C++ and C interface. I would appreciate feedback, ideas to make it better, use cases and other contributions.

 

Now head over to JonoonDB's website where you can find installation instructions, tutorials, road map and much more.

 

Thanks,

Zarian.

[Wouter van Oortmerssen]

Exciting! Can you tell us a bit more about how e.g. querying on FlatBuffer data works?

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Piyush Mishra]

My goodness, someone actually did it!

[Zarian Waheed]

I am planning to write a section on DB internals that will explain these things in more detail. But the short answer is that we use sqlite vtable mechanism to generate and execute a query plan that is most efficient for a given query. You can get really sophisticated with the vtable implementation e.g. jonoondb will use index data structure if user has created index on a field. If there is no index then it will read the flatbuffers data using reflection. We rely heavily on flatbuffers reflection for accessing the data dynamically. If you are really interested you can look at the jonoondb_vtable implementation. There is a lot of supporting code that goes around it but this essentially has the essence of the SQL functionality in JonoonDB.

You may also find FlatbuffersDocument class to be interesting. This has the code to read flatbuffers data using reflection.

Thanks,

Zarian.

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers...@googlegroups.com.

[N N]

Any response to this ? How query qorks and how fast is that ?

On Monday, January 9, 2017 at 11:44:09 AM UTC-8, Wouter van Oortmerssen wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers...@googlegroups.com.

[agallego]

On Monday, January 9, 2017 at 1:47:07 AM UTC-5, Zarian Waheed wrote:

I am very happy and excited to announce JonoonDB. An open source multi model database that supports flatbuffers as a native storage format.

 

Website: www.jonoondb.com

 

The database is in alpha stage. There is still some way to go before it is feature complete and production ready. But there is enough functionality in place that it can be useful for many use cases. Some of the features available today are: 

1. Store your flatbuffers data reliably in jonoondb (protected against process and OS crashes).
   
2. Retrieve the flatbuffers data.
   
3. Query the flatbuffers data using SQL. Full SQL support, select, group by, order by, where clause, aggregations etc.
   
4. Index flatbuffers data for even faster query execution.
   
5. Compress the data as you are writing it in the database.
   
6. No need for ORM since the database natively supports flatbuffers.
7. View flatbuffers data inside JonoonDB using jonoondb_cli tool. A  command line tool for interacting with databases. 

Currently JonoonDB is available as a library with C++ and C interface. I would appreciate feedback, ideas to make it better, use cases and other contributions.

I think an architecture document would be helpful to understand where does sqlite3 comes in. How are you projecting the indexes / etc. 

In general I don't see you using an async IO api (libaio / nor an abi substitution / dma-{read,write}). This is a major concern to me since i care about tail latencies.

As a library, i don't see any guarantees about failures. When do you fsync the data. What happens during recovery of the database file. How is integrity checked. etc. 

cool project.