DL and transactions?

27 views
Skip to first unread message

Xi Liu (Ant Financial)

unread,
May 13, 2016, 4:09:17 AM5/13/16
to distributedlog-user
Does DL provide built-in transactions? Took a look at the code, the answer seems to be no... Any plan to support this? it would be useful for some transactional use cases.

- Xi

Leigh Stewart

unread,
May 13, 2016, 10:46:15 AM5/13/16
to Xi Liu (Ant Financial), distributedlog-user
Depends what you mean. You can batch writes to ensure all writes in the batch are written (or not) atomically.
There is no manual rollback mechanism, and RDBMS-like transaction behavior is not supported.

Can you explain what you're looking for? 

On Fri, May 13, 2016 at 1:09 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
Does DL provide built-in transactions? Took a look at the code, the answer seems to be no... Any plan to support this? it would be useful for some transactional use cases.

- Xi

--
You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-...@googlegroups.com.
To post to this group, send email to distribut...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/e6325174-42ee-4ae5-a230-eae0ecfb0471%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Xi Liu (Ant Financial)

unread,
May 13, 2016, 12:34:23 PM5/13/16
to distributedlog-user
How large a batch that DL supports? If the user writes more than the size of batch, it doesn't actually work. If there are transactions supported, user can do transactional writes. That would simply user logic.

- Xi

Leigh Stewart

unread,
May 13, 2016, 12:37:34 PM5/13/16
to Xi Liu (Ant Financial), distributedlog-user
Sure I agree. 1MB is the max batch size atm. 

How would transactions work for you ideally?

Do you just need to write arbitrarily large batches atomically? Or do you need something more advanced?  

To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/d4fd75fb-8014-4baf-b5c4-2c589dc5dadb%40googlegroups.com.

Xi Liu (Ant Financial)

unread,
May 13, 2016, 10:07:12 PM5/13/16
to distributedlog-user
Be able to publish arbitrarily large batches atomically is a basic requirement. The consumer should see all the data together or not see them at all. We could handle it in our logic, but that would be a bit complicated. I feel if DL supports such ability, it would be super useful.

- Xi

On Friday, May 13, 2016 at 9:37:34 AM UTC-7, lstewart wrote:
Sure I agree. 1MB is the max batch size atm. 

How would transactions work for you ideally?

Do you just need to write arbitrarily large batches atomically? Or do you need something more advanced?  
On Fri, May 13, 2016 at 9:34 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
How large a batch that DL supports? If the user writes more than the size of batch, it doesn't actually work. If there are transactions supported, user can do transactional writes. That would simply user logic.

- Xi


On Friday, May 13, 2016 at 7:46:15 AM UTC-7, lstewart wrote:
> Depends what you mean. You can batch writes to ensure all writes in the batch are written (or not) atomically.
> There is no manual rollback mechanism, and RDBMS-like transaction behavior is not supported.
>
>
> Can you explain what you're looking for? 
>
>
> On Fri, May 13, 2016 at 1:09 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
>
> Does DL provide built-in transactions? Took a look at the code, the answer seems to be no... Any plan to support this? it would be useful for some transactional use cases.
>
>
> - Xi
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-user+unsub...@googlegroups.com.
>
> To post to this group, send email to distribut...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/e6325174-42ee-4ae5-a230-eae0ecfb0471%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-user+unsub...@googlegroups.com.
> To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-user+unsub...@googlegroups.com.
>
> To post to this group, send email to distribut...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/e6325174-42ee-4ae5-a230-eae0ecfb0471%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-user+unsub...@googlegroups.com.

To post to this group, send email to distribut...@googlegroups.com.

Sijie Guo

unread,
May 14, 2016, 3:05:52 AM5/14/16
to Xi Liu (Ant Financial), distributedlog-user
This is an interesting requirement. However I think either it would be super useful, or just there is not lots applications will use it. I am hesitate at adding such features as part of the core logic. It would be nice thinking in a different way - "if this necessary need to be in core library" "can this be done by wrapper". If it can be done by a wrapper, you might be able to start with a simple wrapper over the core library and iterate from there.

- Sijie 

On Fri, May 13, 2016 at 7:07 PM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
Be able to publish arbitrarily large batches atomically is a basic requirement. The consumer should see all the data together or not see them at all. We could handle it in our logic, but that would be a bit complicated. I feel if DL supports such ability, it would be super useful.

- Xi
On Friday, May 13, 2016 at 9:37:34 AM UTC-7, lstewart wrote:
Sure I agree. 1MB is the max batch size atm. 

How would transactions work for you ideally?

Do you just need to write arbitrarily large batches atomically? Or do you need something more advanced?  
On Fri, May 13, 2016 at 9:34 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
How large a batch that DL supports? If the user writes more than the size of batch, it doesn't actually work. If there are transactions supported, user can do transactional writes. That would simply user logic.

- Xi


On Friday, May 13, 2016 at 7:46:15 AM UTC-7, lstewart wrote:
> Depends what you mean. You can batch writes to ensure all writes in the batch are written (or not) atomically.
> There is no manual rollback mechanism, and RDBMS-like transaction behavior is not supported.
>
>
> Can you explain what you're looking for? 
>
>
> On Fri, May 13, 2016 at 1:09 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
>
> Does DL provide built-in transactions? Took a look at the code, the answer seems to be no... Any plan to support this? it would be useful for some transactional use cases.
>
>
> - Xi
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-...@googlegroups.com.
>
> To post to this group, send email to distribut...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/e6325174-42ee-4ae5-a230-eae0ecfb0471%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-...@googlegroups.com.

To post to this group, send email to distribut...@googlegroups.com.

On Friday, May 13, 2016 at 9:37:34 AM UTC-7, lstewart wrote:
Sure I agree. 1MB is the max batch size atm. 

How would transactions work for you ideally?

Do you just need to write arbitrarily large batches atomically? Or do you need something more advanced?  
On Fri, May 13, 2016 at 9:34 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
How large a batch that DL supports? If the user writes more than the size of batch, it doesn't actually work. If there are transactions supported, user can do transactional writes. That would simply user logic.

- Xi


On Friday, May 13, 2016 at 7:46:15 AM UTC-7, lstewart wrote:
> Depends what you mean. You can batch writes to ensure all writes in the batch are written (or not) atomically.
> There is no manual rollback mechanism, and RDBMS-like transaction behavior is not supported.
>
>
> Can you explain what you're looking for? 
>
>
> On Fri, May 13, 2016 at 1:09 AM, Xi Liu (Ant Financial) <xi.li...@gmail.com> wrote:
>
> Does DL provide built-in transactions? Took a look at the code, the answer seems to be no... Any plan to support this? it would be useful for some transactional use cases.
>
>
> - Xi
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-...@googlegroups.com.
>
> To post to this group, send email to distribut...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/e6325174-42ee-4ae5-a230-eae0ecfb0471%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-...@googlegroups.com.

To post to this group, send email to distribut...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "distributedlog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distributedlog-...@googlegroups.com.

To post to this group, send email to distribut...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/distributedlog-user/feb44ae4-8c79-4fc0-9578-8cf7c8b87a8f%40googlegroups.com.

Xi Liu

unread,
May 16, 2016, 2:20:57 AM5/16/16
to Sijie Guo, distributedlog-user
Thanks for explanation. That makes sense.

- Xi
Reply all
Reply to author
Forward
0 new messages