Production Experience + Landscape?

[Rahul Tongia]

Hi Rustam,

Firstly, thanks for the awesome ElasticInbox. I have been playing with elasticInbox for building our email storage layer. We like you intend to store a lot of emails here. In your London presentation, you mention you are running 4 nodes for nearly 30M+ emails. 

A few questions on cluster setup in the example?

1.  Do you have Cassandra and ElasticInbox running on the same machine? or Do you split those?

2.  For blob storage (in private cloud) whats your recommended Blob Store specially when it comes to performance and maintenance? Recently, I am looking at RiakCS, but very few have prod experience with it.

Thanks,

Rahul

[John Wiesel]

Hi Rahul,
I am not an elastic inbox developer but more a user and also currently looking into elasticinbox. Regarding your questions:
1. Cassandra can be quite CPU intensive. You can start with Cassandra and Elasticinbox on 1 node and have at least 1 extra Cassandra node. Then create the elasticinbox keyspace with a  replication factor of 2 (or higher), so you can retire the Cassandra node if you see too high CPU loads and move the Cassandra node to a dedicated machine.
2. Have you considered openstack storage? There are quite professional hosters running openstack storage in a production environment. (http://www.openstack.org/user-stories/#Cloud%20Hosting%20/%20MSP%20/%20Telco)

Hope that helps!
John

[Rahul Tongia]

Thanks John, This is really helpful. We reconsidered our requirements and close to go for MySQL for metadata storage. The reason was more due to team competence and relatively small size as a startup. We in some ways, deferred the scaling pain for future if we are successful and big for flexibility in product design that MySQL schema gives over Cassandra. 

I will keep posted the group with our progress and challenges.

Thanks a lot.

--
You received this message because you are subscribed to a topic in the Google Groups "ElasticInbox" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticinbox/Bze48cSSP3c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticinbox...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Rustam Aliyev]

Hi Rahul,

Somehow I missed your email.

1.  Do you have Cassandra and ElasticInbox running on the same machine? or Do you split those?

Currently production setup we run grew to 6x C* nodes and 2x EI nodes. EI nodes run side-by-side with C*. Cassandra is not CPU hungry unless you run compaction.
 

2.  For blob storage (in private cloud) whats your recommended Blob Store specially when it comes to performance and maintenance? Recently, I am looking at RiakCS, but very few have prod experience with it.

We evaluated Ceph Rados and OpenStack Object Store. Unfortunately, both of them were too immature and didn't perform well for small files. Therefore we've added hybrid mode storage in EI 0.4 (see https://github.com/elasticinbox/elasticinbox/wiki/Blob-Storage#hybrid-storage) and encryption. This allowed us to move to public cloud (Azure, which outperformed S3 in speed test) and keep the costs low.
 

Thanks,

Rahul

 

[Rahul Tongia]

Hi Rustam,

Thanks a lot for the details and building this awesome project. Our datamodel is a little more complex than just storing emails because we need to synchronise all the recipient mailboxes of the shared email folders (which could run to high double digits), making C* datamodel inflexible for us without a lot of indexes. We are building the v1 on MySQL as we already use it for the frontend. We have designed it with sharding in mind, so hopefully we can scale when we need to. As a startup, sometimes speed trumps scale :).

For blob storage, as we intend to keep the mails in our cluster (mostly for sync performance reasons amongst various boxes). We are still looking at RiakCS blobstore or a DFS like xteemFS, The OpenStack things do not look stable enough yet. For smaller files (as defined by you), we might consider storing them in DB directly together. But really need to test the need for it.

Once we have something presentable, we would be happy to share our experience as with ElasticInbox group, as we learnt quiet a lot conceptually from that. 

Thanks a lot!

Cheers,

Rahul

 

--

[Rustam Aliyev]

Shared folders are definitely interesting, but not currently on our priority list. Good luck with your startup and do keep us updated please!

--
You received this message because you are subscribed to the Google Groups "ElasticInbox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticinbox...@googlegroups.com.