Is there a way to find all PersistentEntities of certain type?

[Joo Lee]

Hello guys,

Currently, whenever I need to send some commands to the PersistentEntities in bulk, I use repository to query the id of all the PersistentEntities, then I do entityRef(id).ask to send commands one by one.

i.e. 

portfolioRepository.getAllPortfolioIds().map{

    id=> 

           entityRef(id).ask(RunDailyBatch(LocalDate.now))

}

But I just want to know if there is a way I can get all entities of certain type, i.e. PortfolioEntity that Lagom has.

I do think the chance of such feature exist is very low considering the Lagom's distributed nature, but just double checking.

Thanks,

Joo

[Tim Moore]

Joo,

You are correct, there is no API for getting all of the entities by type.

I would also explore whether there's an alternative approach that might be better. Getting everything and looping through it this way can have a few issues.

First, what happens if this loop fails partway through? You'll have sent the command to some entities but not to others, and there isn't really an easy way to resume from the middle of a failed batch.

Second, this will load all of your entities into RAM in quick succession, so you'll need to have enough RAM available to hold your entire data set resident and will have periodic spikes in resource consumption.

What triggers the code above? Will it resume scheduled runs if the node it runs on fails? How do you make sure it doesn't run more than once per day if it restarts on another node?

What's happening in the RunDailyBatch command? At the very least, I would suggest spreading the load out over the day, which might mean putting the "next run" time into your read-side repository database based on the time of each entity's creation. Then you would be loading all of the repositories that are past due, rather than absolutely all of them. At the end of the daily batch, you would emit an event that updates the repository and sets the next run to +1 day.

Cheers,

Tim

--
You received this message because you are subscribed to the Google Groups "Lagom Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lagom-framework+unsubscribe@googlegroups.com.
To post to this group, send email to lagom-framework@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lagom-framework/89a1a3f4-6be4-421c-b0fa-e9d9824e5c05%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tim Moore

Senior Engineer, Lagom, Lightbend, Inc.

tim....@lightbend.com
+61 420 981 589
Skype: timothy.m.moore

[James Roper]

Hi Joo,

The concerns that Tim raises with regards to failure tolerance are important - I would probably create a read side view that tracks all the entity IDs, and that same view would also be used to keep the status of running the daily batch on each entity.  That view would give you a simple way of querying all the persistent entity ids.

That said, while Lagom doesn't give you an API to get all the persistent entity IDs, Akka persistence does, and it's very straight forward to drop down to Akka persistence APIs.  We use it here:

https://github.com/lagom/online-auction-scala/blob/master/user-impl/src/main/scala/com/example/auction/user/impl/UserServiceImpl.scala#L40

You should never use it for something that's going to happen frequently (the above is a perfect example of what not to do), but for running a daily batch task it's fine (forgetting the resilience issues associated with doing that).

Regards,

James

To view this discussion on the web visit https://groups.google.com/d/msgid/lagom-framework/CAApsmORCZBLqUDtW6mLB0fKuYGPPV6XUwVor8Zj3-8YQDoMDYg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

James Roper
Software Engineer

Lightbend – Build reactive apps!
Twitter: @jroper