distributed-deep-learning

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

Distributed Keras Engine, Make Keras faster with only one line of code.

Learn applied deep learning from zero to deployment using TensorFlow 1.8+

A Portable C Library for Distributed CNN Inference on IoT Edge Clusters

sensAI: ConvNets Decomposition via Class Parallelism for Fast Inference on Live Data

Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.

Distributed training of DNNs • C++/MPI Proxies (GPT-2, GPT-3, CosmoFlow, DLRM)

RocketML Deep Neural Networks

SHADE: Enable Fundamental Cacheability for Distributed Deep Learning Training

Intel® End-to-End AI Optimization Kit

Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.

🚨 Prediction of the Resource Consumption of Distributed Deep Learning Systems

TensorFlow (1.8+) Datasets, Feature Columns, Estimators and Distributed Training using Google Cloud Machine Learning Engine

Decentralized Asynchronous Training on Heterogeneous Devices

Eager-SGD is a decentralized asynchronous SGD. It utilizes novel partial collectives operations to accumulate the gradients across all the processes.

Scalable NLP model fine-tuning and batch inference with Ray and Anyscale

WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.

Java based Convolutional Neural Network package running on Apache Spark framework

This repository contains the implementation of a wide variety of Deep Learning Projects in different applications of computer vision, NLP, federated, and distributed learning. These projects include university projects and projects implemented due to interest in Deep Learning.

Distributed deep learning framework based on pytorch/numba/nccl and zeromq.