Atomic space renaming in tarantool and "create space" on insert?

[Kir]

Hello,

I have numerous clients writing to space "space1". I have to move data from "space1" to another storage. So, before looping over "space1" tuples and converting them to CSV I have to rename this space to stop any modifications of this space.

At the same time, clients are continue writing to "space1", but it was already renamed so they'll be getting error that space do not exists.

Some possible solutions:

a) Atomic rename. So before renaming "space1" I can create space with similar structure, for example "space_tmp". Next I should atomically rename "space1" > "space_export", "space_tmp" > "space1".

b) "Create space" on insert could be a real help (like in redis, we don't care if key exists or no, any write operation on key will create it automatically).

c) Any another ways of how this problem could be solved?

Thank you!

[Konstantin Osipov]

* Kir <kirz...@gmail.com> [16/06/03 10:11]:

> Hello,
>
> I have numerous clients writing to space "space1". I have to move data from
> "space1" to another storage. So, before looping over "space1" tuples and
> converting them to CSV I have to rename this space to stop any
> modifications of this space.
> At the same time, clients are continue writing to "space1", but it was
> already renamed so they'll be getting error that space do not exists.

We simply planned to implement transactional DDL in the future.

To create a backup of a space, you shouldn't need to rename it.
We do backups in the snapshot thread using a consistent read view.

We have not exposed this feature in the iterator API, but this is
also a possibility.

Finally, we plan to implement replication filters, which would
allow replicas to subscribe to changes to a single space. If you
send JOIN command with a single-space filter, you get a
consistent backup of a single space.

Anyway, all the above features are planned for future releases.

I can't offer you a solution for 1.6, except using the approach
for range iteration over a space: select a range of keys from the
primary key, back it up, select another range, beginning from
the last key from the previous range. This will not make it a
single transaction, but will backup all your tuples.

Here's an example:

https://github.com/tarantool/expirationd/blob/master/expirationd.lua#L87

Please take a look at how last_id is used.

--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.org - www.twitter.com/kostja_osipov