Build Stuff

In the modern world, the speed of making decisions and predicting trends based on data is becoming essential. Beating competitors through savvy near-realtime analytics, deriving automatic actions or decision options from massive, never-ending streams of data will become more and more important in the very near future. Sensors, web behaviour and traffic, mobile applications, internet of things are just a few examples of where such data can come from. Data like this needs to be taken in with adequate speed, reliability and scalability. Then, this data needs to be analysed, aggregated and eventually enriched. After that, a fast analysis needs to be done for ad-hoc decisions and actions. Further, this data needs to be reliably stored and quickly retrieved for different comparisons and long-term-predictions and exploratory analysis. All this reveals a huge challenge to technology. It's too easy these days - due to the massive choice and short time-to-market - to make a wrong technological decision which will influence the success of the solution and thus lead to the promise of continuous, fast decision making and automatic actions based on data not to be kept. Streaming technologies, vertical vs. horizontal grow, cascading, pipelining and aggregation, in-memory-processing, reliable storage, fast retrieval and not to forget the analytical complexity have to be considered when building such a solution. And it's always good to have a minimal, yet powerful abstraction and a good alternative for any of these technology-chain members. In this talk I will go through the topic and give the audience advice to which approaches and tools to use, how well they fit into the landscape of near-realtime analytics, rapid decision making and action automation based on massive data streams.