Re: Bedrock

[David Barrett]

Hi, thanks for writing!  Responses inline:

On Sun, Jul 19, 2020 at 12:01 PM Thiery Laverdure <thi...@spacestud.io> wrote:

1. Does Bedrock only run in-memory or can it run from disk like normal Sqlite? (My mind was blown that you are running servers that are probably at 800GB by now)

It runs from disk just like normal SQLite.  For performance reasons obviously it works best if you are able to cache it entirely in RAM because SQLite memory-maps the entire database, so this will minimize the number of major page faults (ie, disk reads).  So we have gobs of RAM, and use NVME drives.  But yes, it's fundamentally just a normal SQLite database under the good.  (And our instance is substantially larger than 800GB, btw.)

1. Have you ever tried or been asked if Bedrock can run in a serverless environment? Recently AWS added EFS mounting to Lambda. With this addition I've read that Sqlite can run on Lambda but you'd eventually run into issues at scale with file locking. Wondering if Bedrock could solve this?

Hm, I don't know if Bedrock would be the best tool for any kind of Lambda serverless environment.  Bedrock is explicitly intended to make SQLite into a classic networked database server (albeit, one optimized for WAN replication and fault tolerance), so I'm not sure what it would mean to run a server in a serverless environment.  For example, the first thing Bedrock does when started is try to connect to every other server in the cluster, elect a leader, synchronize down the latest commits, etc.  I would think in a Lamda environment you'd want to skip all that.  Furthermore, if you are using EFS, I think you'd just want to use Lambda and SQLite direct -- EFS is taking care of the network synchronization after all.  As for locking, I'm not sure it would be a problem: so long as you use WAL mode (to enable multiple writers), and BEGIN CONCURRENT (which does page-level locking), it might work.

That said, we've had extremely disappointing performance with Lambda in general -- and any database hosted on a networked file system is going to be pretty dicey.  I think I'd suggest using EC2 to set up a classic Bedrock cluster (ideally, 3 nodes, each in different regions), and then use Lambda (again in those same 3 regions) to just handle your application code in a serverless way, making normal network calls to Bedrock.

Then again, if you're using Lamda, you're totally dependent on AWS anyway, so you might just use some AWS native database, like RDS, or Firebase, etc.

<rant>

Regardless, I'm probably not the best person to ask for AWS advice: I think the whole thing is massively overrated, incredibly overengineered, and astonishingly expensive for the low performance and erratic reliability it provides.  I think it's like Amazon has done some kind of Jedi mind trick to convince everyone that it is somehow faster, cheaper, more reliable, and easier than self-hosting your own Linux servers -- despite every single one of those claims being (so far as I can tell) patently false.  I'd suggest you save yourself the effort and take advantage of the enormous cost, reliability, and performance advantages of buying and colocating Linux servers (seriously, they cost almost nothing; hardware is basically free anymore) -- and focus your innovation on the product you are shipping to customers.  They don't care at all if you are using some fancy auto-scaling serverless environment: it's a nightmare to get set up even at small scales, and will almost certainly fail you at large scales (assuming you ever get there, which the simple odds suggest you won't -- specially if you are overcomplicating your tech stack out of the gate).

</rant>

Good luck!

-david