second-order-optimization

PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)

A C++ interface to formulate and solve linear, quadratic and second order cone problems.

An implementation of PSGD Kron second-order optimizer for PyTorch

Distributed K-FAC Preconditioner for PyTorch

FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)

This repository implements FEDL using pytorch

PyTorch implementation of the Hessian-free optimizer

Tensorflow implementation of preconditioned stochastic gradient descent

Implementation of PSGD optimizer in JAX

Hessian-based stochastic optimization in TensorFlow and keras

Compatible Intrinsic Triangulations (SIGGRAPH 2022)

This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.

Federated Learning using PyTorch. Second-Order for Federated Learning. (IEEE Transactions on Parallel and Distributed Systems 2022)

Minimalist deep learning library with first and second-order optimization algorithms made for educational purpose

LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms

Subsampled Riemannian trust-region (RTR) algorithms

Prototyping of matrix free Newton methods in Julia