
انتقل إلى وضع عدم الاتصال باستخدام تطبيق Player FM !
AI Evaluations Crash Course in 50 Minutes (2025) | Hamel Husain
Manage episode 508886594 series 3621237
Today, I want to share a new episode with Hamel Husain.
Hamel has trained 2,000+ PMs and engineers from companies like OpenAI, Anthropic, and Google on how to run AI evals. In my new episode, he shares a free master class on how to build evals for a real AI agent in just 50 minutes using a simple spreadsheet. I learned a lot from Hamel and I think you will too.
Hamel and I talked about:
(00:00) What the most valuable part of evals is
(01:25) Live walkthrough: Analyzing 100 real production traces
(09:50) Creating the eval criteria using a simple spreadsheet
(24:44) Why binary pass/fail ratings beat 1-5 scores every time
(28:52) The agreement metric trap that fools most PMs
(30:08) True positive and negative rates explained
(36:00) How to set up continuous evals in production
Get the takeaways: https://creatoreconomy.so/p/ai-evaluations-crash-course-in-50-minutes-hamel-husain
Where to find Hamel:
Website: https://hamel.dev/
📌 Subscribe to this channel – more interviews coming soon!
77 حلقات
Manage episode 508886594 series 3621237
Today, I want to share a new episode with Hamel Husain.
Hamel has trained 2,000+ PMs and engineers from companies like OpenAI, Anthropic, and Google on how to run AI evals. In my new episode, he shares a free master class on how to build evals for a real AI agent in just 50 minutes using a simple spreadsheet. I learned a lot from Hamel and I think you will too.
Hamel and I talked about:
(00:00) What the most valuable part of evals is
(01:25) Live walkthrough: Analyzing 100 real production traces
(09:50) Creating the eval criteria using a simple spreadsheet
(24:44) Why binary pass/fail ratings beat 1-5 scores every time
(28:52) The agreement metric trap that fools most PMs
(30:08) True positive and negative rates explained
(36:00) How to set up continuous evals in production
Get the takeaways: https://creatoreconomy.so/p/ai-evaluations-crash-course-in-50-minutes-hamel-husain
Where to find Hamel:
Website: https://hamel.dev/
📌 Subscribe to this channel – more interviews coming soon!
77 حلقات
सभी एपिसोड
×مرحبًا بك في مشغل أف ام!
يقوم برنامج مشغل أف أم بمسح الويب للحصول على بودكاست عالية الجودة لتستمتع بها الآن. إنه أفضل تطبيق بودكاست ويعمل على أجهزة اندرويد والأيفون والويب. قم بالتسجيل لمزامنة الاشتراكات عبر الأجهزة.