adam-optimizer

On the Variance of the Adaptive Learning Rate and Beyond

Deep learning library in plain Numpy.

This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"

CS F425 Deep Learning course at BITS Pilani (Goa Campus)

ADAS is short for Adaptive Step Size, it's an optimizer that unlike other optimizers that just normalize the derivative, it fine-tunes the step size, truly making step size scheduling obsolete, achieving state-of-the-art training performance

Google Street View House Number(SVHN) Dataset, and classifying them through CNN

Lion and Adam optimization comparison

PyTorch/Tensorflow solutions for Stanford's CS231n: "CNNs for Visual Recognition"

Reproducing the paper "PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks" for the ICLR 2019 Reproducibility Challenge

Implemented Adam optimizer in python

Toy implementations of some popular ML optimizers using Python/JAX

A collection of various gradient descent algorithms implemented in Python from scratch

This library provides a set of basic functions for different type of deep learning (and other) algorithms in C.This deep learning library will be constantly updated

A compressed adaptive optimizer for training large-scale deep learning models using PyTorch

The project aimed to implement Deep NN / RNN based solution in order to develop flexible methods that are able to adaptively fillin, backfill, and predict time-series using a large number of heterogeneous training datasets.

Modified XGBoost implementation from scratch with Numpy using Adam and RSMProp optimizers.

Lookahead optimizer ("Lookahead Optimizer: k steps forward, 1 step back") for tensorflow

[Python] [arXiv/cs] Paper "An Overview of Gradient Descent Optimization Algorithms" by Sebastian Ruder

From linear regression towards neural networks...

Implementation of Adam Optimization algorithm using Numpy