The schedule for the webinar is the following:
• 18:30 – 18:45: Opening the meeting
• 18:45 – 19:05: Using cutting-edge open-source technologies to build one of the biggest industrial Data Lake of the World – Allan Sene
• 19:05 – 19:15: Q&A
• 19:15 – 19:35: Intro to FastAPI – Sebastián Ramírez
• 19:35 – 19:45: Q&A
• 19:45: Closing
See you there!
Using cutting-edge open-source technologies to build one of the biggest industrial Data Lake of the World
On this talk, we will discuss how DataSprints is using cutting-edge open-source technologies, such as Dremio and dbt, to build one of the biggest industrial Data Lakes of the World, serving more than 200 reports, processing more than 300Gb and 2000+ columns datasets in near-realtime (8s), keeping the cloud costs very low, and by very low we mean, VERY LOW. We will pass through the architecture now in production, all the challenges and lessons learned on this project
Allan Sene has a background in Computer Science and Statistics, having worked with Data since 2010, from genetic data transformation to complex industrial data lakes.
With DataSprints, Allan is Designing, Developing, and Implementing Data Engineering/Analytics projects on several companies, from startups like MaxMilhas to big corporates like Accenture, Telefonica, and Vallourec.
Allan built and still developing Data Hackers, the biggest Data community in South America. Helping professionals to change their careers, spreading knowledge about Data Engineering, Analytics, and Data Science disciplines and best practices. There are more than 30k professionals and enthusiasts on Data Hackers network, consuming curated content by him and his partners through newsletters, podcasts, and blogs
Intro to FastAPI
Learn how to create an API ready for production in very little time using FastAPI… explained with memes. Your API will have automatic validation, documentation based on standards, high performance, and several other features. All this, having editor support including autocompletion everywhere. In this talk, you will learn what FastAPI can do, and how it could benefit you. You will see how to declare the data you want to receive in each request using standard Python type annotations. Including path parameters, query parameters, body payloads with JSON, etc. You will also see how to use simple, standard, Python type annotations to declare complex JSON body payloads with deeply nested structures, and get automatic data validation, serialization, and documentation.
I’m Sebastián Ramírez (tiangolo), the creator of FastAPI, Typer, and other open-source tools. I’m currently a developer at Explosion in Berlin, Germany.