Treffer: gpauloski/kfac-pytorch: v0.4.0

Complete refactor of kfac-pytorch See Pull Requests #38, #40, #41, and #42. DevOps changes kfac requires torch>=1.8 and Python >=3.7 tox used for testing environments and automation pre-commit updated. Major changes include prefer single-quotes, mypy, flake8 plugins Switch to setup.cfg for package metadata and tox/flake8/mypy/coverage configuration Add requirement-dev.txt that contains all dependencies needed to run the test suite Code quality and testing Complete type annotations for all code Passes mypy Separated testing utilities and unit tests into testing/ and tests/ respectively Expansive unit testing suite that achieves 100% code coverage New testing utilities include wrappers for simulating distributed environments and small test models Added end-to-end training tests small unit test (included in pytest) that checks loss decreases when training with K-FAC MNIST integration test (not run with pytest) that verifies training with K-FAC achieves higher accuracy kfac package improvements KFAC layers separated from PyTorch module wrappers KFACBaseLayer handles general K-FAC computations and communications for an arbitrary layer ModuleHelper implementations provide a unified interface for interacting with supported PyTorch modules Provides methods that return the size of the factors for the layer so the size of factors can be determined prior to training Provides methods for getting the current gradients, updating the gradients, and computing the factors from the intermediate data Each KFACBaseLayer instance is passed a ModuleHelper instance corresponding to the module in the model being preconditioned Removed broken LSTM/RNN/Embedding layer support Module registration utilities moved out of the preconditioner class and into the kfac.layers.register module Replaced the comm module with the distributed module that provide a more exhaustive set of distributed communication utilties All communication ops now return futures to the return value to allow more aggressive asynchronous communication Added ...