This month's beCraft meeting is next Wednesday, March 12th, from 6-8PM at the Center for Open Science (directions below)
I'm really excited about this meeting. Andrew Montalenti is leading a discussion on stream architectures, a.k.a., log-centric or lambda architectures. This is an interesting architecture that's been getting some buzz lately, and we're fortunate to have an experienced practitioner among us. I've included background links below.
Stream Architectures for Real-time Data Analysis
Some of the biggest issues at the center of analyzing large amounts of data are query flexibility, latency, and fault tolerance. Modern technologies that build upon the success of “big data” platforms, such as Apache Hadoop, have made it possible to spread the load of data analysis to commodity machines, but these analyses can still take hours to run and do not respond well to rapidly-changing data sets.
A new generation of data processing platforms -- which Parse.ly internally calls “stream architectures” -- have converted data sources into streams of data that can be processed and analyzed in real-time. This has led to the development of various distributed real-time computation frameworks (e.g. Apache Storm) and high-performance data integration technologies (e.g. Apache Kafka). Andrew Montalenti (Parse.ly's co-founder & CTO) will give an overview from what he has seen in the field, and how his team applied this knowledge to a backend re-implementation at Parse.ly.
The Center for Open Science
is located in the Downtown Business Center (DBC) of the Omni Hotel
at 210 Ridge McIntire Road, Suite 500
. The DBC faces Ridge/McIntire Rd. opposite Staples/McDonalds. You can park in the Omni parking garage – it's free, just bring your ticket inside for validation. You can enter through the glass doors in the front. Or, walk through the lobby of the Omni Hotel past the concierge desk, ballrooms, and through a set of double doors.
Some background links:
Please RSVP if you are planning to attend so we can get the right amount of pizza.
See you next Wednesday!