I ♥ Logs: Apache Kafka, Stream Processing, and Real-time Data

Jay Kreps
LinkedIn

This talk will discuss how logs and stream-processing can form a backbone for data flow, ETL, and real-time data processing. It will describe the challenges and lessons learned as LinkedIn built out its real-time data subscription and processing infrastructure. It will also discuss the role of real-time processing and its relationship to offline processing frameworks such as MapReduce.

Slides: Download the slides for this presentation in PDF format.

Videos:

• Join the live presentation. Wednesday November 5, 4:15-5:30.  Requires Microsoft Windows Media player.
• View video by lecture sequence. Fall 2014 only, HTML5. Available after 8PM on the days of the lecture.
• View Video on YouTube.

About the speaker:

Jay Kreps is a Principal Staff Engineer at LinkedIn where he is the lead architect for online data infrastructure. He is among the original authors of several open source projects including a distributed key-value store called Project Voldemort, a messaging system called Kafka, and a stream processing system called Samza.

Contact information:

Jay Kreps
Twitter: @jaykreps