Descendint Through a Crowded Valley - Benchmarkiing Deep learning optimizers

Posted on Oct 11, 2020

From https://www.youtube.com/watch?v=DiNzQP7kK-s

A paper that does some benchmarking what optimizers to use.

One thing I liked is the list of all the optimizers and schedulers there are.

Conclusion

::: anki 34307583-9dce-48e7-b0f4-e4e59059368e Which is the best optimizer for DL? Adam or SGD with momentum #Flashcard :::