What's the stateSpout used for?

[Jimmy Jim]

I tracked the process of submit topology in storm,and found nothing to
do with the stateSpoutMap.
So,What's the stateSpout userd for?

[Nathan Marz]

The state spout isn't fully implemented yet, so you should ignore it for now. When it's finished, it will allow you to synchronize external sources of state into a topology. 

Other things have been higher priority, delaying its development (especially the upcoming TransactionalSpout).

--
Twitter: @nathanmarz
http://nathanmarz.com

[Richards Peter]

Hi Nathan,

This mail is not related to the issue reported in the mail's subject. But it is related to you reply to the previous mail.

I would like to know the release plans for TransactionalSpout. By when can we expect a release with this feature? This is a feature that all of us are looking forward to.

Richards.

[Nathan Marz]

I'm finishing up development/testing of it. So the abstraction itself is planned to be done within the next week, but I was going to wait to release it until I had a concrete implementation of it (going to try Apache Kafka for first attempt).

Are you interested in having access to TransactionalSpout before I produce the Kafka implementation?

[Richards Peter]

Hi Nathan,

We are interested to have an access to TransactionSpout. We would really like to know how it would look like. We would also like to have a documentation(even though it is simple) on the api exposed like how the flow of control takes place.

Thanks and regards,
Richards.

[Nathan Marz]

I'll make a development release when TransactionalSpout is ready.

TransactionalSpout is actually one piece of the larger abstraction of "transactional topologies", which is essentially a pipelined and generalized incremental batch processing abstraction on top of Storm. If you want to see what it will look like, see this package in the transactional-spout branch:

https://github.com/nathanmarz/storm/tree/transactional-spout/src/jvm/backtype/storm/transactional

The key things to look at there are:

1. TransactionalTopologyBuilder

2. ITransactionalSpout

3. ITransactionalBolt

I'm only planning some minor API changes to what you see there.

Of course, this will all be thoroughly documented when it's released.

[Richards Peter]

Thanks Nathan for sharing the link. Lets check it out.

Richards Peter.