Event Store

125 views
Skip to first unread message

Cemal Erdem

unread,
Jan 13, 2020, 6:46:27 AM1/13/20
to event...@googlegroups.com
Hi,

Is there a way to create multiple databases in Eventstore? I mean is it possible to create 2 different databases in Eventstore. The think that I want to do is that I will read all events from one database and write them to another databases in eventstore as .NET client.

I want this because I have to migration to my eventstore database. Also I have to change all events type name from my old event store. When I searched some articles the peoples says that I can use copy and transform techniques. With this techniques can I read my old events and modified them. After that I will write them to the new eventstore database. 

Do you think this is good approach? Is there any way to change events type name? Alsa anyone know about copy and transformation techniques because I find nothing about this technique. How can I do that.

I will be appricate if you help me 

Thank you 





- Bu e-posta mesaj ve ekleri sadece gönderildigi kisi veya kuruma özeldir. Dogru aliciya ulasmamis olmasi halinde, bu mesajin baska bir aliciya yonlendirilmesi, kopyalanmasi veya kullanilmasi yasaktir.

- This e-mail and any attachments transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any forwarding, copying or use of the information is prohibited.

- ‘Protel Bilgisayar AŞ olarak kişisel verilerinizin gizliliğini önemsiyoruz. Bizimle paylaştığınız kişisel verileriniz yasal mevzuat kapsamında işlenmektedir. Üçüncü kişilere ilişkin bilgi paylaşımı yapmanız halinde yasal sorumluluk size aittir. 6698 sayılı Kişisel Verilerin Korunması Kanunu kapsamında oluşturduğumuz aydınlatma metnimize www.protel.com.tr üzerinden ulaşabilirsiniz.’ .

‘At Protel Bilgisayar Inc, we herewith proclaim to have proper technological and physical security measures in place that protect your personal and business data. Data shared with us is protected in accordance to legal regulations. In case of third-party data sharing, the reader accepts that they have legal responsibility. You can visit www.protel.com.tr to get more information and read our legal statement in accordance with the Turkish Personal Data Protection Law no. 6698.’

Rodolfo Yabut

unread,
Jan 13, 2020, 2:54:43 PM1/13/20
to event...@googlegroups.com
re: multiple databases w/ full replication -- In version 5, you can use clones. Version 6 has replication for this sort of thing.

But what it sounds like is you want to do a migration from one event store to Event Store (the specific database for this group). In that case, you can do a full replay from your old event store, create a subscriber, and have that subscriber fire the events into Event Store. It would be a similar process if you wanted to create an event store with only a subset of the events from the primary event store.

--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event-store...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/CAFbEux6f1H4T0nK5ih%3DOgaGCt1WpU470whyrf%2B9h2WH08B8NzQ%40mail.gmail.com.

Cemal Erdem

unread,
Jan 13, 2020, 2:59:48 PM1/13/20
to event...@googlegroups.com
Hi

Yes the thing that I want to do is that we have a running project and storing data into eventstore. Our domain has changed and we have to modify the events type name so that's why I have to migrate. So which techniques I can use for the update the events type name do you think. 

Thank you 

13 Oca 2020 Pzt 22:54 tarihinde Rodolfo Yabut <rodolf...@gmail.com> şunu yazdı:

Greg Young

unread,
Jan 13, 2020, 3:04:39 PM1/13/20
to event...@googlegroups.com
I wrote ... a book about this :)

Not to punt you off but it's more in depth than what I would write here.


To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/CAFbEux51u4sKtAa5kjZ64evrm1EoXmiuAGe5%3DnnN5qBnYYUmRw%40mail.gmail.com.

john nicholas

unread,
Jan 13, 2020, 3:04:45 PM1/13/20
to event...@googlegroups.com
It's just a projection.
Subscribe, read and write the other out.

Your complexity isn't there imo. it is how you move from subscriptions on old to new and not miss or double process anything.

To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/CAFbEux51u4sKtAa5kjZ64evrm1EoXmiuAGe5%3DnnN5qBnYYUmRw%40mail.gmail.com.

Cemal Erdem

unread,
Jan 13, 2020, 3:07:52 PM1/13/20
to event...@googlegroups.com
Hi

Oh yes I read all the pages in that book. And I decided to use Copy and Transformation techniques. That's why I ask how to use 2 databases in eventstore. Do you think copy and paste is good approach for my problem. 

By the way it's really nice book. 

Thank you 

13 Oca 2020 Pzt 23:04 tarihinde Greg Young <gregor...@gmail.com> şunu yazdı:
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/CAC9RQtgmqXt2DqpOHkqe%2BMcnqg_iRiYxmkNRS0-wyEgjB0-aVw%40mail.gmail.com.

Rodolfo Yabut

unread,
Jan 13, 2020, 8:49:17 PM1/13/20
to Event Store
picking this up right now. also interested in the unpublished event sourcing ebook :)
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

Cemal Erdem

unread,
Jan 14, 2020, 12:40:35 AM1/14/20
to event...@googlegroups.com
Yes interested :) 

14 Oca 2020 Sal 04:49 tarihinde Rodolfo Yabut <rodolf...@gmail.com> şunu yazdı:
To unsubscribe from this group and stop receiving emails from it, send an email to event-store...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/dbc09a0d-ae46-4127-a16b-d6bf45bfc948%40googlegroups.com.

Cemal Erdem

unread,
Jan 14, 2020, 3:21:29 AM1/14/20
to event...@googlegroups.com

By the way do you know any migration tool for evetstore ? Because i only find one migration tool but it’s for PostgreSQL

 

 

 

Windows 10 için Posta ile gönderildi

 

Kimden: Cemal Erdem
Gönderilme: Tuesday, January 14, 2020 8:40 AM
Kime: event...@googlegroups.com
Konu: Re: Event Store

Peter Hageus

unread,
Jan 14, 2020, 8:39:33 AM1/14/20
to event...@googlegroups.com
You can’t have separate databases in EventStore. You can either start a separate instance, or migrate your streams in the same instance (later deleting/truncating the old ones)

/Peter

Yorick Laupa

unread,
Jan 14, 2020, 8:57:21 AM1/14/20
to Event Store
Have a look at the Cerberus administration tool announcement[1] !

John Lazos

unread,
Jan 14, 2020, 9:08:25 AM1/14/20
to Event Store
I would say don't overthink along the idea of "migration". I feel that DB migration is a concept in a world where state is the source of truth, i.e. relational databases, document databases. Having to keep state consistent while migrating to a larger db instance is historically a pain, hence the hefty implications of the word "migration".

Because data in event store is immutable and append-only, it has some interesting properties that make traditional db migration a moot point. If you run eventstore in a cluster and regularly snapshot the file storage, i'm not sure we'd need a separate cluster. All "migration" would mean would be to copy around files between disks.

But if you want a separate cluster, e.g. for failover in case of a load balancing issue, you might be able to look into subscriptions? https://eventstore.org/blog/getting-started-part-3-subscriptions/

As for copy-transform, i would just create a new stream and play the old stream into it, as other posts suggest.

Greg Young

unread,
Jan 14, 2020, 9:19:22 AM1/14/20
to event...@googlegroups.com
Just to add a bit on the "multiple databases". This was originally planned to support at the beginning. Over time it seemed less and less ... useful. Not even wanted.

It is a substantial piece of work to put in today. Note the large amounts of work have literally nothing to do with actually implementing it!!! I could literally start in this today and have a PR by end of week at latest its pretty straight forward *internally*! However ... consider for a moment how you would get an atom feed from Database db1? The URI would need to change from /streams/foo/ to /db1/streams/foo, no? This would also require a change on every atom client. The TCP api would need changes as well no? What about all the TCP clients.... Do you make no db provided go to a default? How does the default work? Lets imagine this in v5 what happens from a v4 client?

If not clear its not the internally supporting that would even be difficult. It is the API change that would literally hit *almost every* message if this makes more sense? Even changing the messages/uris/etc would not be *horrific* its then ... every client needs updating ...

The change itself is relatively trivial < +- a week or two. Its that the fan out is extremely high and we don't even support everything it would fan out to. 

Note: we have not discussed ... documentation, tests, supporting tools, etc?

There has however been long going discussion over supporting this. The difficulties associated are not technical in nature if that makes sense?


To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/249B2B54-76C7-4505-A049-3074E52BD8AC%40gmail.com.


--
Studying for the Turing test

Stephen Tung

unread,
Jan 14, 2020, 7:56:07 PM1/14/20
to Event Store
Greg, I know where you're coming from and no doubt this is more of a business problem than a technical one.

Despite that I'd still like to +1 on the multiple DB idea. For me it's because I don't want to mix concepts from different bounded contexts into the same DB to pollute the language. My only choice right now is to create separate instances which can be more difficult to maintain and more expensive on the IT side. No biggie - just makes it slightly harder for me to justify the adoption of ES where I'm at right now.

-Stephen


On Tuesday, January 14, 2020 at 10:19:22 PM UTC+8, Greg Young wrote:
Just to add a bit on the "multiple databases". This was originally planned to support at the beginning. Over time it seemed less and less ... useful. Not even wanted.

It is a substantial piece of work to put in today. Note the large amounts of work have literally nothing to do with actually implementing it!!! I could literally start in this today and have a PR by end of week at latest its pretty straight forward *internally*! However ... consider for a moment how you would get an atom feed from Database db1? The URI would need to change from /streams/foo/ to /db1/streams/foo, no? This would also require a change on every atom client. The TCP api would need changes as well no? What about all the TCP clients.... Do you make no db provided go to a default? How does the default work? Lets imagine this in v5 what happens from a v4 client?

If not clear its not the internally supporting that would even be difficult. It is the API change that would literally hit *almost every* message if this makes more sense? Even changing the messages/uris/etc would not be *horrific* its then ... every client needs updating ...

The change itself is relatively trivial < +- a week or two. Its that the fan out is extremely high and we don't even support everything it would fan out to. 

Note: we have not discussed ... documentation, tests, supporting tools, etc?

There has however been long going discussion over supporting this. The difficulties associated are not technical in nature if that makes sense?


/Peter

To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

Greg Young

unread,
Jan 14, 2020, 9:21:34 PM1/14/20
to event...@googlegroups.com
Its not even "business" ...its breaking change. It would literally break every client!

To unsubscribe from this group and stop receiving emails from it, send an email to event-store...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/6b37e37a-fcef-4915-9d45-3b7f47bdf240%40googlegroups.com.

Rickard Öberg

unread,
Jan 14, 2020, 9:59:11 PM1/14/20
to event...@googlegroups.com
If there was such a concept as "default database", then current feeds would point to that. If the client understands multitenancy it can add prefixes to get to a specific one. With the TCP clients, would it be possible to add something like "select database" to shift to something other than the default?

Would this work as a way to support existing clients, while allowing newer clients to take advantage of multidatabase support?

cheers, Rickard

Mohamed Abed

unread,
Jan 15, 2020, 5:56:25 PM1/15/20
to Event Store
I agree, adding multi-database / multi-tenancy will help with many use cases. Defaulting to main database should resolve backward compatibility issues.
I can see huge benefit of having multiple stores to support multi-tenancy.


--
Studying for the Turing test

--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

Greg Young

unread,
Jan 15, 2020, 6:11:47 PM1/15/20
to event...@googlegroups.com
It would in fact be *rare* to want to do a db per tenant, no? How often do you do with this say SQL Server? I mean if count = 5 sure ... count = 50 eeek ... count = 5000?!?!?!

To unsubscribe from this group and stop receiving emails from it, send an email to event-store...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/e461b858-7c72-4673-898e-3334e3476bac%40googlegroups.com.

Mohamed Abed

unread,
Jan 15, 2020, 6:37:00 PM1/15/20
to Event Store
That is one of the reasons I don't do SQL ;)

I am running MongoDB for read model, and it is setup as one database per tenant .. and it is hugely simplifying things .. Tenants come and go (specially with trial account). When tenant is expired simple deletion of their MongoDB does the job. I hope I can do the same with EventStore.

On Thursday, January 16, 2020 at 10:11:47 AM UTC+11, Greg Young wrote:
It would in fact be *rare* to want to do a db per tenant, no? How often do you do with this say SQL Server? I mean if count = 5 sure ... count = 50 eeek ... count = 5000?!?!?!

john nicholas

unread,
Jan 16, 2020, 7:22:54 AM1/16/20
to event...@googlegroups.com
If I wanted multiple dB

Or say schemas then I'd use a stream naming convention.

Personally I like all the conversations I no longer have to have about managing change/ownership/responsibility of multiple databases on one instance.






To unsubscribe from this group and stop receiving emails from it, send an email to event-store...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/6b37e37a-fcef-4915-9d45-3b7f47bdf240%40googlegroups.com.

Greg Young

unread,
Jan 16, 2020, 12:41:34 PM1/16/20
to event...@googlegroups.com
There are uses for multiple dbs ... A good one would be isolation. I want to 100% ensure that service A can't read service B's data, this can be handled via security etc but could be better handled with isolation. Another would be in dealing more easily with backups/restores of a service. It can be done today with multiple instances but the resources overhead in particular for memory is fairly high.

To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/CAFri78qp%3DdUbawRv6EgJkTuYMT0Tdosfbn7Gf0daKBgPRMA08Q%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages