))
Where'd My Gradient Go? It Vanished!
Manage episode 446714678 series 3605861
المحتوى المقدم من Brian Carter. يتم تحميل جميع محتويات البودكاست بما في ذلك الحلقات والرسومات وأوصاف البودكاست وتقديمها مباشرة بواسطة Brian Carter أو شريك منصة البودكاست الخاص بهم. إذا كنت تعتقد أن شخصًا ما يستخدم عملك المحمي بحقوق الطبع والنشر دون إذنك، فيمكنك اتباع العملية الموضحة هنا https://ar.player.fm/legal.
This video discusses the vanishing gradient problem, a significant challenge in training deep neural networks. The speaker explains how, as a neural network becomes deeper, gradients—measures of how changes in network parameters affect the loss function—can decrease exponentially, leading to a situation where early layers of the network are effectively frozen and unable to learn. This problem arises because common activation functions like the sigmoid function can produce very small derivatives, which compound during backpropagation. The video then explores solutions like using different activation functions (like ReLU) and architectural changes (like residual networks and LSTMs) to mitigate this issue.
Watch the video: https://www.youtube.com/watch?v=ncTHBi8a9uA&pp=ygUSdmFuaXNoaW5nIGdyYWRpZW50
59 حلقات
مشاركة
