How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022)

Data on Kubernetes Community

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

Abstract

Working with Terabytes of data is a major challenge for organizations both in terms of architecture and cost. In recent years, a new paradigm has emerged in the world of Big Data, that is, implementing the entire architecture for processing massive data from a microservices approach. This architecture proved to be scalable, efficient, with excellent performance, failsafe, portable between different cloud providers and at a lower cost than conventional architectures. In this talk, I will present how my team managed to implement this architecture based on Kubernetes and what were the main gains we had with the innovation.