Maximum size database rqlite with VM ubuntu 2G ram ?

[xuan nhat]

Hi,

I have 2 questions about rqlite:

1. Maximum size database rqlite with VM ubuntu 2G ram ? (Regularly free 50% ram before use rqlite), with default uses an in-memory database.

2. with rqlite uses an in-memory database by default then:

Since the Raft log is the authoritative store for all data, and it is written to disk by each node, an in-memory database can be fully recreated on start-up  

How long does it take the node to recover from the Raft log to back memory with the 1G database?

[Philip O'Toole]

On Tue, Dec 22, 2020 at 1:31 AM xuan nhat <xuan...@gmail.com> wrote:

Hi,

I have 2 questions about rqlite:

1. Maximum size database rqlite with VM ubuntu 2G ram ? (Regularly free 50% ram before use rqlite), with default uses an in-memory database.

I have no hard data to share with you, this question is difficult to answer in a simple fashion. A computer will usually page to disk if it runs out of physical RAM. You should load-test your system, and observe when performance decreases (slow queries, CPU load goes up, etc) as the database grows.

 

2. with rqlite uses an in-memory database by default then:

Since the Raft log is the authoritative store for all data, and it is written to disk by each node, an in-memory database can be fully recreated on start-up  

How long does it take the node to recover from the Raft log to back memory with the 1G database?

It depends on a lot of factors including the size of data in the log, your computer's resources, speed of your disk, etc. You must perform this kind of testing yourself, since everyone's environment is different.

 

--
You received this message because you are subscribed to the Google Groups "rqlite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rqlite+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/621ab87d-fe0d-45c4-aa7f-64505e8cee74n%40googlegroups.com.

[xuan nhat]

Thanks for your answer,

Rqlite is very interesting, I am looking to use rqlite in applications.

I want to use rqlite to save temperature logs from IOT device over http of rqlite, Once every 5 second will inserted, but I'm worried that the database will get bigger and full of ram. If I use rqlite with -on-disk (performance issue), is there any other disadvantage? Do not use ram will no full memory ram ?

[Philip O'Toole]

On Wed, Dec 23, 2020 at 12:24 AM xuan nhat <xuan...@gmail.com> wrote:

Thanks for your answer,

Rqlite is very interesting, I am looking to use rqlite in applications.

I want to use rqlite to save temperature logs from IOT device over http of rqlite, Once every 5 second will inserted, but I'm worried that the database will get bigger and full of ram. If I use rqlite with -on-disk (performance issue), is there any other disadvantage? Do not use ram will no full memory ram ?

Use -on-disk and you shouldn't have any memory issues. Performance won't be as good, but it may not matter for your application. You just need to test it, using your expected load.

On Tuesday, December 22, 2020 at 10:32:33 PM UTC+7 Philip O'Toole wrote:

On Tue, Dec 22, 2020 at 1:31 AM xuan nhat <xuan...@gmail.com> wrote:

Hi,

I have 2 questions about rqlite:

1. Maximum size database rqlite with VM ubuntu 2G ram ? (Regularly free 50% ram before use rqlite), with default uses an in-memory database.

I have no hard data to share with you, this question is difficult to answer in a simple fashion. A computer will usually page to disk if it runs out of physical RAM. You should load-test your system, and observe when performance decreases (slow queries, CPU load goes up, etc) as the database grows.

 

2. with rqlite uses an in-memory database by default then:

Since the Raft log is the authoritative store for all data, and it is written to disk by each node, an in-memory database can be fully recreated on start-up  

How long does it take the node to recover from the Raft log to back memory with the 1G database?

It depends on a lot of factors including the size of data in the log, your computer's resources, speed of your disk, etc. You must perform this kind of testing yourself, since everyone's environment is different.

 

--
You received this message because you are subscribed to the Google Groups "rqlite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rqlite+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/621ab87d-fe0d-45c4-aa7f-64505e8cee74n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "rqlite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rqlite+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/c9f7c341-60aa-431f-a614-ecdc2d51f141n%40googlegroups.com.

[xuan nhat]

Thanks for your reply,

I still have one question, with  -on-disk when computer (vm) restart then rqlite will rqlite will start up faster vs in-memory ?

because when rqlite startup will run on disk database immediately, other than memory database will fully recreated on start-up from Raft log ,Is that true ?

[Philip O'Toole]

Whether you run in-memory, or on-disk, any existing SQLite database is deleted and recreated from scratch on restart. This is done to ensure the Raft Log and SQLite database are completely consistent after a restart.

To view this discussion on the web visit https://groups.google.com/d/msgid/rqlite/05b0d50b-27b8-4d9b-9b51-40b97d5747c4n%40googlegroups.com.

[xuan nhat]

Thanks, I understand how rqlite works.

[Philip O'Toole]

FWIW, v.5.8.0 now includes the size of the SQLite database in RAM, in the status output. Check for the key named "db_size", in the "sqlite3" block. This is basically the SQLite page size multiplied by the page count.

Philip