Emit in projection doesn't create/write to new stream

36 views
Skip to first unread message

Farhan Nasim

unread,
Apr 11, 2020, 10:04:22 PM4/11/20
to Event Store
Hi All,
  1. I am using EventStore 5.0.1-1 official Docker image.
  2. I have only one stream (called title:A4CAABEE-D8A1-4581-9444-61950FB7584E, with 2 events) and only the system projections enabled.
  3. I would like to re-write the stream to a new stream called new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E.

Here follows my OneTime projection called migration:

fromStream('title:A4CAABEE-D8A1-4581-9444-61950FB7584E')
.when(function (s, e) {
    var newStreamName = 'new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E';
    emit(newStreamName, e.eventType, e.data, e.metadata);
})

The detailed definition of the projection as returned by ES server for clarity.

➜  ~ curl -i http://localhost:2113/projection/migration/query%3Fconfig=yes -H "Accept: application/json" -u "admin:changeit"
HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, PUT, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:33:11 GMT
Content-Length: 574
Keep-Alive: timeout=15,max=100
{
  "msgTypeId": 262,
  "name": "migration",
  "query": "fromStream('title:A4CAABEE-D8A1-4581-9444-61950FB7584E')\r\n.when(function (s, e) {\r\n    var newStreamName = 'new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E';\r\n    emit(newStreamName, e.eventType, e.data, e.metadata);\r\n})\r\n",
  "emitEnabled": true,
  "definition": {
    "allEvents": true,
    "categories": [],
    "events": [],
    "streams": [
      "title:A4CAABEE-D8A1-4581-9444-61950FB7584E"
    ],
    "options": {}
  },
  "outputConfig": {
    "resultStreamName": "$projections-migration-result"
  }
}%

The projection seems to have finished properly.

➜  ~ curl -i http://localhost:2113/projections/any -H "Accept: application/json" -u "admin:changeit"
    {
      "coreProcessingTime": 0,
      "version": 2,
      "epoch": -1,
      "effectiveName": "migration",
      "writesInProgress": 0,
      "readsInProgress": 0,
      "partitionsCached": 1,
      "status": "Completed/Stopped/Writing results",
      "stateReason": "",
      "name": "migration",
      "mode": "OneTime",
      "position": "Phase: 1 (completed)",
      "progress": 100.0,
      "lastCheckpoint": "Phase: 1 (completed)",
      "eventsProcessedAfterRestart": 1,
      "statusUrl": "http://localhost:2113/projection/migration",
      "stateUrl": "http://localhost:2113/projection/migration/state",
      "resultUrl": "http://localhost:2113/projection/migration/result",
      "queryUrl": "http://localhost:2113/projection/migration/query%3Fconfig=yes",
      "enableCommandUrl": "http://localhost:2113/projection/migration/command/enable",
      "disableCommandUrl": "http://localhost:2113/projection/migration/command/disable",
      "checkpointStatus": "",
      "bufferedEvents": 0,
      "writePendingEventsBeforeCheckpoint": 0,
      "writePendingEventsAfterCheckpoint": 0
    }
 
➜  ~ curl -i http://localhost:2113/projection/migration/state -H "Accept: application/json" -u "admin:changeit"         HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:47:35 GMT
Content-Length: 306
Keep-Alive: timeout=15,max=100
{"eventBody":"{\"eventVersion\":1,\"occurredOn\":\"1586655651224\",\"titleId\":{\"id\":\"A4CAABEE-D8A1-4581-9444-61950FB7584E\"},\"copyAddedCount\":2,\"oldCopyCount\":1,\"newCopyCount\":3}","eventId":-1,"occurredOn":"1586655651224","typeName":"com.fnasim.esbookstore.inventory.domain.model.TitleCopyAdded"}%

However, the new stream  new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E is not created.

➜  ~ curl -i http://localhost:2113/streams/new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E -H "Accept: application/json" -u "admin:changeit"
HTTP/1.1 404 Not Found
Access-Control-Allow-Methods: POST, DELETE, GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Content-Type: text/plain; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:50:59 GMT
Content-Length: 0
Keep-Alive: timeout=15,max=100

Did I miss anything or got something wrong in the projection definition? Further details are available on request. Please help.

Yorick Laupa

unread,
Apr 12, 2020, 5:07:43 AM4/12/20
to event...@googlegroups.com


Looks like you didn’t allow the projection to emit events. Adding emit=true in the URL query parameter list should fix your problem.

On 12 Apr 2020, at 04:04, Farhan Nasim <f.na...@gmail.com> wrote:


--
You received this message because you are subscribed to the Google Groups "Event Store" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event-store...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/event-store/128decf4-5c71-417e-99cc-d0bec88b488e%40googlegroups.com.

Farhan Nasim

unread,
Apr 12, 2020, 5:47:09 AM4/12/20
to Event Store
Hi Yorick,

Thanks for the reply. However, I can see emit had been very much enabled.

Here follows the projection query response:
And, just in case, the projection configs view:

Screenshot_14.png


Please check.
To unsubscribe from this group and stop receiving emails from it, send an email to event...@googlegroups.com.

Yorick Laupa

unread,
Apr 12, 2020, 6:07:38 AM4/12/20
to Event Store
Hi Farhan,

I just saw in one of your cURL results that emit is set to true. However, emitting events only work with continuous projections. Based on your previous image, it appears you configured a one-time projection.
Please switch to continuous projection and see if it solves your problem.

Farhan Nasim

unread,
Apr 12, 2020, 6:23:59 AM4/12/20
to Event Store
Hi Yorick,

1. Yes, last projection had been One-Time.

2. 
emitting events only work with continuous projections

Didn't know that. Any reason for that constraint?

3. Have created a new Continuous projection (migration_new) with the same operation, however, still no good.

➜  ~ curl -i http://localhost:2113/projection/migration_new/query%3Fconfig=yes -H "Accept: application/json" -u "admin:changeit"
HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, PUT, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 09:40:30 GMT
Content-Length: 582
Keep-Alive: timeout=15,max=100

{
  "msgTypeId": 262,
  "name": "migration_new",
  "query": "fromStream('title:A4CAABEE-D8A1-4581-9444-61950FB7584E')\r\n.when(function (s, e) {\r\n    var newStreamName = 'new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E';\r\n    emit(newStreamName, e.eventType, e.data, e.metadata);\r\n})\r\n",
  "emitEnabled": true,
  "definition": {
    "allEvents": true,
    "categories": [],
    "events": [],
    "streams": [
      "title:A4CAABEE-D8A1-4581-9444-61950FB7584E"
    ],
    "options": {}
  },
  "outputConfig": {
    "resultStreamName": "$projections-migration_new-result"
  }
}%

➜  ~ curl -i http://localhost:2113/projections/any -H "Accept: application/json" -u "admin:changeit"

    {
      "coreProcessingTime": 1,
      "version": 2,
      "epoch": -1,
      "effectiveName": "migration_new",
      "writesInProgress": 0,
      "readsInProgress": 0,
      "partitionsCached": 1,
      "status": "Running",
      "stateReason": "",
      "name": "migration_new",
      "mode": "Continuous",
      "position": "title:A4CAABEE-D8A1-4581-9444-61950FB7584E: 1",
      "progress": 100.0,
      "lastCheckpoint": "title:A4CAABEE-D8A1-4581-9444-61950FB7584E: -1",
      "eventsProcessedAfterRestart": 2,
      "checkpointStatus": "",
      "bufferedEvents": 0,
      "writePendingEventsBeforeCheckpoint": 0,
      "writePendingEventsAfterCheckpoint": 0
    }

➜  ~ curl -i http://localhost:2113/projection/migration_new/state -H "Accept: application/json" -u "admin:changeit"
HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
ES-Position: {"$s":{"title:A4CAABEE-D8A1-4581-9444-61950FB7584E":1}}
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 09:47:01 GMT
Content-Length: 306
Keep-Alive: timeout=15,max=100

{"eventBody":"{\"eventVersion\":1,\"occurredOn\":\"1586655651224\",\"titleId\":{\"id\":\"A4CAABEE-D8A1-4581-9444-61950FB7584E\"},\"copyAddedCount\":2,\"oldCopyCount\":1,\"newCopyCount\":3}","eventId":-1,"occurredOn":"1586655651224","typeName":"com.fnasim.esbookstore.inventory.domain.model.TitleCopyAdded"}%
➜  ~ curl -i http://localhost:2113/streams/new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E -H "Accept: application/json" -u "admin:changeit"
HTTP/1.1 404 Not Found
Access-Control-Allow-Methods: POST, DELETE, GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Content-Type: text/plain; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 09:49:47 GMT
Content-Length: 0
Keep-Alive: timeout=15,max=100

Yorick Laupa

unread,
Apr 12, 2020, 6:51:22 AM4/12/20
to Event Store
Your script seems wrong. Please use that one instead:

fromStream('title:A4CAABEE-D8A1-4581-9444-61950FB7584E')
    .when({
        $any: function (s, e) {
            var newStreamName = 'new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E';
            emit(newStreamName, e.eventType, e.data, e.metadata);
        }
    })

Farhan Nasim

unread,
Apr 12, 2020, 6:57:51 AM4/12/20
to Event Store
You are absolutely right! With the script corrected, both versions (One-Time and Continuous) works now.

Thanks a lot, Yorick.
Reply all
Reply to author
Forward
0 new messages