[ANN] Khepri 0.2.1 - A tree-like replicated on-disk database library

[Jean-Sébastien Pédron]

Hi!

On behalf of the RabbitMQ team, I'm pleased to announce Khepri 0.2.1! It
is available on:
* GitHub (https://github.com/rabbitmq/khepri/releases/tag/v0.2.1)
* Hex.pm (https://hex.pm/packages/khepri)

Khepri is a tree-like replicated on-disk database library for Erlang and
Elixir. This is a library we plan to use inside RabbitMQ as a
replacement for Mnesia. Indeed, we want to leverage Ra[1], our
implementation of the Raft consensus algorithm, to have a uniform
behavior, especially w.r.t. network partitions, across all types of data
managed by RabbitMQ.

The release notes of 0.2.0 [2] describe the highlights of Khepri 0.2.x
in detail, but I would like to quickly talk about the biggest one:
triggers and stored procedures!

Triggers are a mechanism to execute an anonymous function automatically
following some events.

Currently supported events are changes made to the tree: nodes were
created, updated or deleted. In the future, it could support Erlang
process monitoring, node monitoring, and so on. Triggers are registered
using an event filter which, as its name suggests, takes care of
filtering the event which should execute the associated function.

The anonymous function behind a trigger is stored in the database as the
payload of tree node. This function is called a stored procedure. Before
it is stored, the function is extracted like transaction functions are.
However, there are no restrictions on what it can do, unlike transaction
functions.

If you have any comments and feedback, or if you started to play with
the library in one of your projects, please share! I’m looking forward
to listen to your experience!

Thank you very much!

[1] https://github.com/rabbitmq/ra
[2] https://github.com/rabbitmq/khepri/releases/tag/v0.2.0

--
Jean-Sébastien Pédron

[Frank Muller]

Hi Jean-Sébastien,

Very well done!!!

Anything that can save me (+my team) from Mnesia is always welcome.

Do you provide (throughput / latency) benchmarks?

/F

<jean-sebas...@dumbbell.fr>  :

[Jean-Sébastien Pédron]

On 18/02/2022 11:21, Frank Muller wrote:
> Do you provide (throughput / latency) benchmarks?

Thank you!

No, I didn't make any benchmarking. It looks slower in RabbitMQ, but the
library and the integration are still early stage.

I need to measure it properly and share the result for sure!

--
Jean-Sébastien Pédron

[I Gusti Ngurah Oka Prinarjaya]

>>> The release notes of 0.2.0 [2] describe the highlights of Khepri 0.2.x
>>> in detail, but I would like to quickly talk about the biggest one:

>>> triggers and stored procedures!

Triggers in an RDBMS database are a nightmare. I never use trigger in MySQL / PostgreSQL / etc RDBMS.

I worried the trigger feature in Khepri will also become a nightmare. Please reconsider adding a trigger feature.

[Ilya Khaprov]

Hey,

You are welcome to ignore triggers here as well

Ilusha

---

From: erlang-questions <erlang-quest...@erlang.org> on behalf of I Gusti Ngurah Oka Prinarjaya <okapri...@gmail.com>
Sent: Friday, February 18, 2022 3:36:31 PM
To: Jean-Sébastien Pédron <jean-sebas...@dumbbell.fr>
Cc: Erlang <erlang-q...@erlang.org>
Subject: Re: [ANN] Khepri 0.2.1 - A tree-like replicated on-disk database library

 

[Jean-Sébastien Pédron]

On 18/02/2022 15:36, I Gusti Ngurah Oka Prinarjaya wrote:
> Triggers in an RDBMS database are a nightmare. I never use trigger in
> MySQL / PostgreSQL / etc RDBMS.
> I worried the trigger feature in Khepri will also become a nightmare.
> Please reconsider adding a trigger feature.

Thank you for your feedback! I should have explained the use case a bit
more.

Adding, updating or deleting something in the Khepri tree is represented
as a Ra/Raft command. All members of a Khepri cluster are instances of a
state machine. These state machines must apply commands to make progress
with their state and they must all reach the same result given the same
set of commands.

Therefore, transactions in Khepri are not allowed to have side effects
(send a message to an Erlang process, write something to the
disk/network, etc.).

However, sometimes we need those side effects. For instance in RabbitMQ,
when a vhost, a queue, a binding, an internal user, ... is deleted, an
internal event is emitted to notify other processes. This can't happen
inside the transaction code.

One possibility is for the code who requests the transaction to also
emit that event once the transaction is complete. That's ok because it
knows about what it is deleting.

However, some tree nodes are deleted automatically because they were
under another node higher in the tree. In this case, we need an
automatic way to emit that event when these nodes are deleted as a
consequence of something else. This is when triggers are useful.

Here, triggers can't mess with the database content directly. They can
request modifications, but like any caller in the end. Thus they would
go through a Ra command too.

Triggers in Khepri could be compared to a combination between
`erlang:monitor/2` (which would know about tree nodes) and `erlang:apply/2`.

Do you still see that as a source of problems?

--
Jean-Sébastien Pédron

[I Gusti Ngurah Oka Prinarjaya]

Hi Jean-Sébastien,

Thank you so much for your awesome clear explanation.

>>> Triggers in Khepri could be compared to a combination between
>>> `erlang:monitor/2` (which would know about tree nodes) and `erlang:apply/2`.

I feel more comfortable now knowing this comparison.

Thank You

[Jean-Sébastien Pédron]

On 18/02/2022 17:17, I Gusti Ngurah Oka Prinarjaya wrote:
> Thank you so much for your awesome clear explanation.
>
> >>> Triggers in Khepri could be compared to a combination between
> >>> `erlang:monitor/2` (which would know about tree nodes) and
> `erlang:apply/2`.
> I feel more comfortable now knowing this comparison.

Awesome! I will clarify the documentation then to make sure that,
despite having the same name & being kind of the same idea, triggers in
Khepri are not to be confused with triggers in RDBMS.

--
Jean-Sébastien Pédron

[Frank Muller]

Would it be possible to have a syntactic sugar(parse_transform)  for wildcards:

Ret = khepri:get("/stock/wood/_"

instead of:

PathPattern = [stock, wood, #if_node_matches{regex = any}].

Search all brands offering flipflop:

Ret = khepri:get("/shoe/_/flipflop"

It will greatly simplify the interface IMHO.

Best

/F.

<jean-sebas...@dumbbell.fr> a écrit :

[Jean-Sébastien Pédron]

On 19/02/2022 10:39, Frank Muller wrote:
> Would it be possible to have a syntactic sugar(parse_transform)  for
> wildcards:
>
> Ret=khepri:get("/stock/wood/_"
>
> instead of:
>
> PathPattern=[stock,wood,#if_node_matches{regex=any}].

Hi!

String-based paths are already supported:

1> khepri_path:from_string("/shoe/*/flipflop").
[shoe,{if_name_matches,any,undefined},flipflop]

Most functions in the `khepri` module accept string-based paths. The
`khepri_machine` functions don't however.

I have plan to greatly improve the syntax and make `khepri_machine`
accept string-based paths too so the API is uniform.

That said, those paths are parsed at runtime. Would you find it
convenient to have a parse_transform module so they are parsed at
compile time?

--
Jean-Sébastien Pédron

[Frank Muller]

Thanks for the tip! Simply awesome feature. 

parse_transform would be preferable to save CPU cycles IMHO. 

/F.

[Jean-Sébastien Pédron]

On 19/02/2022 16:36, Frank Muller wrote:
> Thanks for the tip! Simply awesome feature.
> parse_transform would be preferable to save CPU cycles IMHO.

Thank you for the suggestion, I filed an issue to remember it.

https://github.com/rabbitmq/khepri/issues/61

--
Jean-Sébastien Pédron

[Jean-Sébastien Pédron]

On 18/02/2022 11:21, Frank Muller wrote:

> Do you provide (throughput / latency) benchmarks?

Hi!

I prepared a first micro-benchmark tool to compare Khepri and Mnesia:
https://github.com/rabbitmq/khepri-benchmark

The README explains what it measures and how it works. In particular, it
only tests operations per second, not latency.

Results are automatically published from GitHub Actions to the following
page (though remember that the CI runner only has 2 CPU cores):
https://rabbitmq.github.io/khepri-benchmark/

--
Jean-Sébastien Pédron

[Frank Muller]

Hi Jean-Sébastien,

Very promising. Would it be possible to test insert (50%) + delete (25%) + update (%25) in the same time? You can vary the numbers of course. 

How about “gets”? Are they faster in Khepri?

Best

F.

sebastie...@dumbbell.fr> a écrit :

On 18/02/2022 11:21, Frank Muller wrote:
> Do you provide (throughput / latency) benchmarks?

Hi!

I prepared a first micro-benchmark tool to compare Khepri and Mnesia:
https://github.com/rabbitmq/khepri-benchmark

The README explains what it measures and how it works. In particular, it
only tests operations per second, not latency.

Results are automatically published from GitHub Actions to the following
page (though remember that the CI runner only has 2 CPU cores):

--
Jean-Sébastien Pédron

[Karl Nilsson]

Frank those figures on ci are quite different from what we saw when run on more powerful systems. It would be great if you could run them yourself and share the results. 

--

Karl Nilsson