Some newbie questions

[ryanc...@gmail.com]

Hello,

Nice to see you guy have an active open source project, great!

Forgive me if my questions is too simple.

1. Is message delivery guaranteed? exactly once and in order?

2. Are there any potential risk in suffering data loss or data corruption?

3. From the blog post [1] it was said that it can handle 18000 per sec, actually I was impressed as ruby is very slow, what are the trick you used to improved performance? (Trade off of reliability?)

Thanks.

[1] http://blog.treasure-data.com/post/13047440992/fluentd-the-missing-log-collector

[Masahiro Nakagawa]

Hi ryanchan,

1.

at-most-once and forwarding model is smilar to Flume NG.

See this image: http://www.slideshare.net/treasure-data/the-basics-of-fluentd/26

2.

You may loss the data when Fluentd daemon is restarted with memory buffer.

3.

Because IO and network cost is the dominant factor on log forwarding.
And we optimizes in_forward / out_forward to reduce the ruby world processing.


Masahiro

> --
> You received this message because you are subscribed to the Google Groups
> "Fluentd Google Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fluentd+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Kazuki Ohta]

Hi,

> 3. From the blog post [1] it was said that it can handle 18000 per sec, actually I was impressed as ruby is very slow, what are the trick you used to improved performance? (Trade off of reliability?)

Actually Fluentd uses cool.io for event loop, and MessagePack for data serialization. Both gems are written mostly by C.

We intentionally write performance sensitive part in C, and other part is written by Ruby for the flexibility.