Getting the same event several times in projected stream - cant understand why

85 views
Skip to first unread message

Julian May

unread,
Oct 6, 2015, 5:17:39 AM10/6/15
to Event Store
Hey

I have a problem with the same events being recieved several times, because the same event appears several times in the projected stream.
How do I prevent this? 

The most problematic/reoccurring event is "RoomstayPlanned", which does occur in a couple of other projected streams... Is the same event being picked up several times because of "fromAll()"?

Anyway - the projection looks like this:

fromAll().when({
   ReservationPlaced: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   RoomstayPlanned: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   RoomstayWasAssignedRoom: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   RoomstayWasUnassignedRoom: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   RoomWasAdded: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   RoomTypeWasAdded: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   PersonCreated: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   CompanyCreated: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   TransientGuestWasAddedToReservation: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   KnownGuestWasRelatedToReservation: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   },
   ReservationBecameGroupReservation: function(s, e) {
      linkTo("Read_RoomAssignments_Assignments", e);
   }
});


And i caught the phenomenon in Postman; here you can see the same event being referenced, but with different "updated" timestamps:


<?xml version="1.0" encoding="utf-8"?>
    <title>Event stream 'Read_RoomAssignments_Assignments'</title>
    <updated>2015-10-06T09:02:22.1623852Z</updated>
    <author>
        <name>EventStore</name>
    </author>
    <entry>
        <title>3@ReservationAR-dd4023a8b50e4a679ddcfb8676cc7942</title>
        <updated>2015-10-06T09:02:22.2393873Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>3@ReservationAR-dd4023a8b50e4a679ddcfb8676cc7942</title>
        <updated>2015-10-06T09:02:22.2373874Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b</title>
        <updated>2015-10-06T09:02:22.2313869Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b</title>
        <updated>2015-10-06T09:02:22.2303871Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-a3e7aa1e0bd9431aa49e508aa6b1fa5b</title>
        <updated>2015-10-06T09:02:22.229387Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-3844357a0659474297a1e18dee0cb8a8</title>
        <updated>2015-10-06T09:02:22.2223871Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-3844357a0659474297a1e18dee0cb8a8</title>
        <updated>2015-10-06T09:02:22.2213873Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-3844357a0659474297a1e18dee0cb8a8</title>
        <updated>2015-10-06T09:02:22.2183869Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>4@ReservationAR-44d8c45cd04e44cab9dfec88e05d11bb</title>
        <updated>2015-10-06T09:02:22.2033866Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
    <entry>
        <title>2@ReservationAR-0de2c58ba7394c97a2d31d518c80b92c</title>
        <updated>2015-10-06T09:02:22.1703856Z</updated>
        <author>
            <name>EventStore</name>
        </author>
        <summary>$&gt;</summary>
    </entry>
</feed>


Any help is much appreciated

/Julian

Julian May

unread,
Oct 7, 2015, 11:11:43 AM10/7/15
to Event Store
No takers? This causes a real problem problem in our write model, and redundant updates (that are less problematic) in our readmodel. Any ideas or hints would be very much appreciated!
What could cause this???

Julian May

unread,
Oct 8, 2015, 8:47:10 AM10/8/15
to Event Store
Is there any way to skip link's? It seems to me that all events, once they reach a projection, is resolved is it was a link. I've been snooping in metadata, streamId etc. for a way to ignore "link"-events, and only linkTo('') for events that is not already a link - because I think that's the reason for the "dobbelganger" events in my projected streams.

James Nugent

unread,
Oct 8, 2015, 8:55:03 AM10/8/15
to Event Store
> I have a problem with the same events being recieved several times, because
> the same event appears several times in the projected stream. How do I prevent
> this?

Your example isn't succinct enough to quickly understand (likely why there are no replies), but if the same event appears multiple times in the stream from which you're running your projection it would be unusual if it didn't get received by the projection multiple times? Moreover, the guarantees are only "at-least-once".

Why is your even appearing multiple times?

Julian May

unread,
Oct 8, 2015, 9:58:58 AM10/8/15
to Event Store
Ok. Maybe I'm having problems with my explenation. I do have something very concrete though:
When I spin up my test-system, emitting events and creating subscriptions, the projection ends in the following state:

{
  "coreProcessingTime": 50,
  "version": 0,
  "epoch": -1,
  "effectiveName": "Read_RoomAssignments_Assignments_Projection",
  "writesInProgress": 0,
  "readsInProgress": 0,
  "partitionsCached": 1,
  "status": "Running",
  "stateReason": "",
  "name": "Read_RoomAssignments_Assignments_Projection",
  "mode": "Continuous",
  "position": "C:2430835/P:2430835; ReservationPlaced: -1; RoomstayPlanned: -1; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: -1; RoomTypeWasAdded: -1; PersonCreated: -1; CompanyCreated: -1; TransientGuestWasAddedToReservation: -1; KnownGuestWasRelatedToReservation: -1; ReservationBecameGroupReservation: -1; ",
  "progress": 100,
  "lastCheckpoint": "C:0/P:-1; ReservationPlaced: -1; RoomstayPlanned: -1; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: -1; RoomTypeWasAdded: -1; PersonCreated: -1; CompanyCreated: -1; TransientGuestWasAddedToReservation: -1; KnownGuestWasRelatedToReservation: -1; ReservationBecameGroupReservation: -1; ",
  "eventsProcessedAfterRestart": 253,
  "checkpointStatus": "",
  "bufferedEvents": 0,
  "writePendingEventsBeforeCheckpoint": 0,
  "writePendingEventsAfterCheckpoint": 0
}


This is wrong (there is for example 18 ReservationPlaced events in the output stream in the output here, 1 one of these being a "dobbelganger")

If we reset the projection, it fixes the issue, and we end at this state:

{
  "coreProcessingTime": 18,
  "version": 1,
  "epoch": 1,
  "effectiveName": "Read_RoomAssignments_Assignments_Projection",
  "writesInProgress": 0,
  "readsInProgress": 0,
  "partitionsCached": 1,
  "status": "Running",
  "stateReason": "",
  "name": "Read_RoomAssignments_Assignments_Projection",
  "mode": "Continuous",
  "position": "C:2426206/P:2426206; ReservationPlaced: 16; RoomstayPlanned: 31; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: 24; RoomTypeWasAdded: 3; PersonCreated: 49; CompanyCreated: 5; TransientGuestWasAddedToReservation: 47; KnownGuestWasRelatedToReservation: 12; ReservationBecameGroupReservation: 2; ",
  "progress": 100,
  "lastCheckpoint": "C:0/P:-1; ReservationPlaced: -1; RoomstayPlanned: -1; RoomstayWasAssignedRoom: -1; RoomstayWasUnassignedRoom: -1; RoomWasAdded: -1; RoomTypeWasAdded: -1; PersonCreated: -1; CompanyCreated: -1; TransientGuestWasAddedToReservation: -1; KnownGuestWasRelatedToReservation: -1; ReservationBecameGroupReservation: -1; ",
  "eventsProcessedAfterRestart": 199,
  "checkpointStatus": "",
  "bufferedEvents": 0,
  "writePendingEventsBeforeCheckpoint": 0,
  "writePendingEventsAfterCheckpoint": 0
}


.... And no dobbelganger-events!

I am at a loss as to why. Why does a "reset" fix it?

Julian May

unread,
Oct 8, 2015, 10:34:33 AM10/8/15
to Event Store
Some more facts:
- The problem is very consistent (the state of the projection ), however the amount of "dobbelgangers", and thus the "eventsProcessedAfterRestart in the stream is not, and "eventsProcessedAfterRestart" varies from 253 to 337 (out of 199 correct) in the handful runs we've done today.
- The problem recurs in similar projections (that simply links events of specific types to another stream), but does not occur in our slightly more complex projections, where we use properties of the events to decide whether or not to link events to another stream.
- The problem is reproducable on other machines (Windows 7, Windows 8 and Windows 10 - same story).
- Everytime, resetting the projection solves it.

Julian May

unread,
Oct 8, 2015, 11:47:06 AM10/8/15
to Event Store
I just realized "succinct" means "brief and to the point" - I can see why this thread quicly suffers TL;DR... 

Greg Young

unread,
Oct 8, 2015, 11:52:37 AM10/8/15
to event...@googlegroups.com
I don't understand what you think is happening, what you expect to
happen, or what the issue is here. I just see projection states
copy/pasted with no context and discussion about events not being
there. Without knowing what the data actually is (and output streams
etc) etc I don't think many will be able to help you. This is just
checkpoints etc.
> --
> You received this message because you are subscribed to the Google Groups
> "Event Store" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to event-store...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

Julian May

unread,
Oct 9, 2015, 5:52:22 AM10/9/15
to Event Store
Ok, sorry for my chaotic stream of information.

To boil it down:

I expect projections to handle each event within scope (in this case, specified EventTypes fromAll() ) exactly once.
When I create and enable the projection Read_RoomAssignments_Assignments_Projection (see Projection.txt), the stream "Read_RoomAssignments_Assignments" contains the entries as specified in StreamBeforeReset.txt. This contains the same events several times. When i reset the projection, the stream ends up like StreamAfterReset.txt, where each event is represented exactly once, as I would expect.

My question is: How can i achive a result like "StreamAfterReset" without resetting my Projection?

StreamBeforeReset.txt
StreamAfterReset.txt
Projection.txt

Julian May

unread,
Oct 9, 2015, 5:52:33 AM10/9/15
to Event Store
StreamBeforeReset.txt
StreamAfterReset.txt
Projection.txt
Reply all
Reply to author
Forward
0 new messages