Train neural networks without backpropagation.
Deepity is an open-source predictive coding engine for C++ and Python. Each neuron learns from its own local error, as the brain is thought to, and training runs on CPUs and NVIDIA GPUs.
- 3.16×
- Faster than ngc-learn at the same accuracy
- 98.22%
- Accuracy on MNIST, trained on a laptop
- CPU + GPU
- From laptop CPUs to NVIDIA GPUs
- MIT
- Free and open source
A different way to train neural networks
Most AI today is trained with backpropagation, which needs a lot of memory and, in practice, GPUs.
Learns locally
Each neuron adjusts from the errors right next to it, the way many neuroscientists think the brain works. No error signal has to travel back through the whole network.
Fits small hardware
Local learning suits edge devices, embedded systems and low-power machines, where a large GPU isn't an option.
Keeps learning
Predictive coding is a good fit for systems that keep learning after deployment, a hard problem for standard networks.
Faster, at the same accuracy
On the standard MNIST handwriting test, Deepity matches the accuracy of the existing predictive coding library, ngc-learn, in about a third of the time. Everything ran on one laptop CPU.
What is inside
Its own engine
The core is C++ and CUDA code written for predictive coding, with Python bindings through nanobind. The compiled extension is under 2MB.
CPU and GPU
SIMD kernels from Google Highway and SLEEF, plus OpenBLAS or Intel MKL, keep it fast on the CPU. A CUDA backend covers NVIDIA GPUs.
Documented and tested
Doxygen docs and runnable examples. GitHub Actions builds the library on Linux and Windows and type-checks the Python package on every push.
Open source
MIT licensed. Made and maintained by one developer.
About the developer ↗Code examples
The same network in Python and C++.
from pydeepity import SimplePCN
from pydeepity.layer import Linear, Sigmoid
# The network from the benchmark: 784 -> 512 -> 512 -> 10
net = SimplePCN(
Linear(784, 512),
Linear(512, 512), Sigmoid(),
Linear(512, 10), Sigmoid(),
batch_size=250,
)
net.configure(learning_rate=0.00373, inference_rate=0.091, lmbda=0.0, optimizer="ADAMW")
# X_train: (N, 784) float32 in [0, 1]. Y_train: (N, 10) one-hot.
# examples/mnist.py loads MNIST and has the full script.
for epoch in range(100):
for b in range(len(X_train) // 250):
X = X_train[b * 250:(b + 1) * 250]
Y = Y_train[b * 250:(b + 1) * 250]
energy = net.train_step_with_projection(X, Y, 20) # 20 settling steps
Building with predictive coding?
Request early access and tell us what you are working on.
Help build it
Deepity is open source under the MIT license. Code, bug reports and docs fixes are all welcome.