|Loggregator||Tammer Saleh||6/25/13 1:35 PM|
Hi to the Clound Foundry community!
I have product management duties for a new project that we've just kicked off -- Loggregator.
I'd like to make sure there's a good avenue for community feedback around the architecture and features for the project. To that end, I've opened a pull request for documenting the new component on docs.cloudfoundry.com. Feel free to watch and/or comment on the pull request.
|Re: [vcap-dev] Loggregator||Matt Stine||6/25/13 1:41 PM|
Looks groovy. :-)
Matt Stine | Community Engineer, Cloud Foundry | Pivotal
901-493-5546 | mstine@goPivotal.com
|Re: [vcap-dev] Loggregator||Mike Youngstrom||6/25/13 1:57 PM|
Awesome! I've been hoping for something like this.
|Re: [vcap-dev] Loggregator||Dr Nic Williams||6/25/13 4:31 PM|
|Re: [vcap-dev] Loggregator||Manikandan Lakshminarayaan||6/25/13 9:03 PM|
Can we not use logstash for aggregating logs instead of a new project or how different is loggregator different from logstash
|Re: [vcap-dev] Loggregator||James Bayer||6/26/13 12:19 AM|
This was one of Tammer's last activities before being out of the office a few days, so let me just jump in and say we did a review of the many of the other solutions in the logging space, there are many. The Cloud Foundry requirements are somewhat unique in that we require a multi-tenant solution that is dynamically configurable as new tenants are created and start sending logs immediately. We have not taken the decision to write something new lightly. If you look at the design documents, I think you'll find that the design is straight-forward. We believe our solution will be as well.
|Re: [vcap-dev] Loggregator||Kristian Øllegaard||6/26/13 7:39 AM|
Did you also take https://github.com/heroku/logplex into consideration? They are more or less working with the same issues as Cloud Foundry.
Den onsdag den 26. juni 2013 09.19.12 UTC+2 skrev James Bayer:
|Re: [vcap-dev] Loggregator||James Bayer||6/26/13 8:18 AM|
Yes, we had a team that spent some hands-on time with LogPlex for several weeks. It's obviously a very capable project used at scale for Heroku and successfully implemented there.
Ultimately we have decided that while the goals are aligned, the team was not able to be very productive with LogPlex. Some of which was due to lack of Erlang familiarity on the team. Some of it was due to difficulty understand it from the at-the-time documentation and test coverage (which I think may have improved some since the first effort started). Some of it was due to perception that it was closely tied to some of Heroku's architecture, which was not as adaptable for our needs. Whether others feel those are accurate or not, these were our realized experiences after digging in on it, hence where we are now.
|Re: [vcap-dev] Loggregator||Mike Youngstrom||6/26/13 10:41 AM|
Is there an approximate time table for integration?
|Re: [vcap-dev] Loggregator||Tammer Saleh||6/27/13 10:34 AM|
We don't have a public timetable, but I can say that this is actively worked on and that we're focused on getting something into production as soon as possible.
|Re: [vcap-dev] Loggregator||Matt Stine||6/27/13 10:40 AM|
Correct me if I'm wrong, but couldn't the Loggregator Drainer hook up to logstash should you desire to use it?
|Re: [vcap-dev] Loggregator||James Bayer||6/28/13 6:54 AM|
Yes, we're trying to build a flexible building block that meets the needs of CF to aggregate logs into a single stream from multiple related sources in a dynamic way that has overload protection. That's the first part. The 2nd part is to provide a very generic drain API and protocol that can be used to retrieve this aggregated log that can then be fed to other places...something like LogStash if you like.