انتقل إلى وضع عدم الاتصال باستخدام تطبيق Player FM !
المدونة الصوتية تستحق الاستماع
برعاية


1 LIVE: Before the Chorus & Open Folk Present: In These Lines feat. Gaby Moreno, Lily Kershaw & James Spaite 33:58
Building Scalable ML Infrastructure at Outerbounds with Savin Goyal
Manage episode 471106946 series 2948506
Machine learning is changing fast, and companies need better tools to handle AI workloads. The right infrastructure helps data scientists focus on solving problems instead of managing complex systems. In this episode, we talk with Savin Goyal, Co-Founder and CTO at Outerbounds, about building ML infrastructure, how orchestration makes workflows easier and how Metaflow and Airflow work together to simplify data science.
Key Takeaways:
(02:02) Savin spent years building AI and ML infrastructure, including at Netflix.
(04:05) ML engineering was not a defined role a decade ago.
(08:17) Modernizing AI and ML requires balancing new tools with existing strengths.
(10:28) ML workloads can be long-running or require heavy computation.
(15:29) Different teams at Netflix used multiple orchestration systems for specific needs.
(20:10) Stable APIs prevent rework and keep projects moving.
(21:07) Metaflow simplifies ML workflows by optimizing data and compute interactions.
(25:53) Limited local computing power makes running ML workloads challenging.
(27:43) Airflow UI monitors pipelines, while Metaflow UI gives ML insights.
(33:13) The most successful data professionals focus on business impact, not just technology.
Resources Mentioned:
https://www.linkedin.com/in/savingoyal/
https://www.linkedin.com/company/outerbounds/
https://airflow.apache.org/
Metaflow -
https://metaflow.org/
Netflix’s Maestro Orchestration System -
https://netflixtechblog.com/maestro-netflixs-workflow-orchestrator-ee13a06f9c78?gi=8e6a067a92e9#:~:text=Maestro%20is%20a%20fully%20managed,data%20between%20different%20storages%2C%20etc.
https://www.tensorflow.org/
PyTorch -
https://pytorch.org/
Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
#AI #Automation #Airflow #MachineLearning
63 حلقات
Building Scalable ML Infrastructure at Outerbounds with Savin Goyal
The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Manage episode 471106946 series 2948506
Machine learning is changing fast, and companies need better tools to handle AI workloads. The right infrastructure helps data scientists focus on solving problems instead of managing complex systems. In this episode, we talk with Savin Goyal, Co-Founder and CTO at Outerbounds, about building ML infrastructure, how orchestration makes workflows easier and how Metaflow and Airflow work together to simplify data science.
Key Takeaways:
(02:02) Savin spent years building AI and ML infrastructure, including at Netflix.
(04:05) ML engineering was not a defined role a decade ago.
(08:17) Modernizing AI and ML requires balancing new tools with existing strengths.
(10:28) ML workloads can be long-running or require heavy computation.
(15:29) Different teams at Netflix used multiple orchestration systems for specific needs.
(20:10) Stable APIs prevent rework and keep projects moving.
(21:07) Metaflow simplifies ML workflows by optimizing data and compute interactions.
(25:53) Limited local computing power makes running ML workloads challenging.
(27:43) Airflow UI monitors pipelines, while Metaflow UI gives ML insights.
(33:13) The most successful data professionals focus on business impact, not just technology.
Resources Mentioned:
https://www.linkedin.com/in/savingoyal/
https://www.linkedin.com/company/outerbounds/
https://airflow.apache.org/
Metaflow -
https://metaflow.org/
Netflix’s Maestro Orchestration System -
https://netflixtechblog.com/maestro-netflixs-workflow-orchestrator-ee13a06f9c78?gi=8e6a067a92e9#:~:text=Maestro%20is%20a%20fully%20managed,data%20between%20different%20storages%2C%20etc.
https://www.tensorflow.org/
PyTorch -
https://pytorch.org/
Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
#AI #Automation #Airflow #MachineLearning
63 حلقات
All episodes
×
1 The Future of Airflow Telemetry at Metyis with Bolke de Bruin 21:55

1 Transforming the Airflow UI for Cloudera’s Users with Shubham Raj 22:28

1 Streamlining Thousands of Data Pipelines at Lyft with Yunhao Qing 19:34

1 Transforming Customer Education in Data Engineering at Astronomer with Marc Lamberti 22:19

1 Embracing Data Mesh and SQL Sensors for Scalable Workflows at lastminute.com with Alberto Crespi 30:09

1 The AI-Ready Pipeline: Reimagining Airflow at Veyer® Logistics with Anu Pabla 23:21

1 Streamlining AI and ML Operations at IBM with BJ Adesoji and Ryan Yackel 24:44

1 Inside the Custom Framework for Managing Airflow Code at Wix with Gil Reich 31:02

1 Modernizing Legacy Data Systems With Airflow at Procter & Gamble with Adonis Castillo Cordero 22:13

1 Building an End-to-End Data Observability System at Netflix with Joseph Machado 38:54

1 Why Developer Experience Shapes Data Pipeline Standards at Next Insurance with Snir Israeli 30:28

1 Data Quality and Observability at Tekmetric with Ipsa Trivedi 22:49

1 Introducing Apache Airflow® 3 with Vikram Koka and Jed Cunningham 27:28

1 Airflow in Action: Powering Instacart's Complex Ecosystem 25:14

1 From ETL to Airflow: Transforming Data Engineering at Deloitte Digital with Raviteja Tholupunoori 27:42
مرحبًا بك في مشغل أف ام!
يقوم برنامج مشغل أف أم بمسح الويب للحصول على بودكاست عالية الجودة لتستمتع بها الآن. إنه أفضل تطبيق بودكاست ويعمل على أجهزة اندرويد والأيفون والويب. قم بالتسجيل لمزامنة الاشتراكات عبر الأجهزة.