 |
Deepity
Ultra-fast, SIMD-accelerated Predictive Coding
|
|
Deepity
Ultra-fast, SIMD-accelerated Predictive Coding
|
| docs | |
| include | |
| deepity | |
| backend | |
| Backend.h | |
| CPUBackend.h | |
| CUDABackend.h | |
| DeviceType.h | |
| IComputeBackend.h | |
| Tensor.h | A device-aware, RAII-managed buffer. Device is fixed at construction – there is no support for moving a live Tensor between CPU and GPU after creation. Loading weights saved on CPU and running them on GPU doesn't need that: read the saved file into a plain std::vector on the host, then construct a DEVICE_GPU Tensor directly from that vector (see the host-data constructors below) – the one-time host-to-device copy happens inside the constructor, and the temporary host vector falls out of scope immediately after. The Tensor itself is a GPU object from the moment it's born |
| layers | |
| ConvPCLayer.h | Convolutional counterpart to DiscriminativePCLayer |
| DirectKPPCLayer.h | Direct Kolen-Pollack predictive coding layer, routed through IComputeBackend |
| DiscriminativePCLayer.h | Defines the single-layer implementation of a PC model |
| FullPCLayer.h | New, standalone PC layer (not derived from DirectKPPCLayer) meant to eventually carry every technique from the paper-combination work – muPC scaling, optional residual connections, DKP direct feedback, AdamW, muP-style per-layer LR scaling, optional iPC, cross-entropy terminal loss – each one OFF by default, opt-in per layer/network, so a plain DKP-PC-equivalent network is just this class with every extra left at its default |
| GaussSeidelPCLayer.h | A PC layer whose settling dynamics follow a Gauss-Seidel (sequential-sweep) update, matching ngc-learn's actual execution order – NOT the Jacobi (fully-synchronous) update SimplePCLayer uses |
| Layer.h | Virtual Layer |
| RBLayer.h | Restricted Boltzmann-style Predictive Coding (PC) Layer |
| SimpleConvPCLayer.h | ConvPCLayer with precision removed, AdamW/Adam support, now routed through IComputeBackend for GPU portability (mirrors SimplePCLayer's own IComputeBackend port) |
| SimplePCLayer.h | DiscriminativePCLayer with precision entirely removed |
| networks | |
| ConvPCNetwork.h | Network-level wrapper for ConvPCLayer, mirroring DiscriminativePCNetwork's structure for the convolutional case |
| DirectKPPCNetwork.h | Orchestrates DirectKPPCLayer's Direct Kolen-Pollack predictive coding (DKP-PC) phases in the correct order – the layer class alone cannot produce a correct train step; the ORDER of the phases across layers is what makes this DKP-PC rather than plain PC, and that only exists at this level |
| DiscriminativePCNetwork.h | Defines the network-level implementation of a PC model |
| FullPCNetwork.h | Orchestrates FullPCLayer's phases, matching DirectKPPCNetwork's exact four-phase DKP-PC structure (see that class's own docs for the full phase breakdown and citation), plus network-level toggles for muPC scaling and residual connections – both OFF by default, so a plain FullPCNetwork with every toggle left alone reproduces DirectKPPCNetwork exactly (confirmed at the layer level: FullPCLayer with a=1.0, useResidual=false is a bit-identical match to DirectKPPCLayer) |
| GaussSeidelPCNetwork.h | Orchestrates GaussSeidelPCLayer's three-phase settling step in the correct order – the layer class alone cannot produce genuine Gauss-Seidel dynamics; the SWEEP ORDER across layers is what makes it Gauss-Seidel rather than Jacobi, and that only exists at this level |
| SimpleConvPCNetwork.h | Convolutional counterpart to SimplePCNetwork, now device-aware (matches DirectKPPCNetwork's own port earlier tonight: DeviceType constructor parameter, backend member, cpuArena/gpuArena split) |
| SimplePCNetwork.h | Network wrapper for SimplePCLayer, mirroring DiscriminativePCNetwork exactly (minus the UpdatePrecision() call, which no longer exists) |
| utils | |
| activations | |
| ActivationDispatch.h | Function-pointer types and ActivationType <-> function dispatch tables shared by every activation family. Forward-declares the concrete activation functions (defined in the sibling Relu.h/Gelu.h/Tanh.h/ Sigmoid.h/Linear.h) so the dispatchers below can take their addresses without depending on inclusion order |
| ActivationMacros.h | The RESTRICT portability macro shared by every activation header. Split out so each per-activation header is includable on its own, without relying on include order through the Activations.h umbrella |
| Gelu.h | GELU (tanh approximation) and its derivative (plain and two-buffer variants), with AVX512/AVX2/SSE paths and a scalar tail |
| Linear.h | The identity activation and its (constant) derivative |
| Relu.h | ReLU and its derivative (plain and two-buffer variants), with AVX512/AVX2/SSE paths and a scalar tail |
| Sigmoid.h | Logistic sigmoid and Elliot ("fast") sigmoid, and their derivatives (plain and two-buffer variants), with AVX512/AVX2/SSE paths and a scalar tail |
| Tanh.h | Tanh and its derivative (plain and two-buffer variants), with AVX512/AVX2/SSE paths and a scalar tail |
| VectorMath.h | SIMD-vectorized exp/log helpers (AVX512/AVX2/SSE with a scalar tail). Currently unused by the rest of the codebase – kept for planned future use (e.g. a vectorized softmax path) rather than deleted |
| Activations.h | Defines the activation functions of a predictive coding model |
| ActivationType.h | Just the ActivationType enum, deliberately split out of Activations.h – zero dependency on <immintrin.h>/<sleef.h>/<omp.h> |
| AdamOptimizer.h | Vectorized AdamW update, operating per-element over an array of weights (or biases) – NOT the single-scalar-averaging design from the earlier draft, which would have collapsed many distinct per-weight gradients into one shared update and thrown away exactly the per-parameter information Adam depends on |
| DeviceMemoryArena.h | A 64-byte-aligned bump-pointer allocator over a single device buffer, mirroring this codebase's host-side MemoryArena, but backed by IComputeBackend::Allocate()/Free() rather than raw cudaMalloc/cudaFree directly – the same allocation path Tensor already uses and CUDAFunctionsVerify already exercises, rather than a second, separate, unverified call site |
| Im2Col.h | Standalone im2col / col2im utilities: convolution expressed as matrix multiplication, so ConvPCLayer can reuse cblas_sgemm rather than needing hand-written conv kernels |
| MemoryArena.h | A 64-byte-aligned bump-pointer allocator over a single heap-allocated block, used to pack every layer's beliefs/errors/ weights/scratch buffers into one contiguous region for cache locality (see the "Contiguous Arena Allocator" design note in the project README). Host-side counterpart to DeviceMemoryArena.h's CUDA version |
| ModelIO.h | Saves/loads a network's compiled weight arena to/from disk, as a JSON header (per-layer shapes/offsets) plus a raw binary float blob |
| Optimize.h | Defines useful optimization functions of a PC model |
| Profile.h | Per-LAYER, phase-level timing accumulator |
| StreamAlignedBatcher.h | C++ port of the Python StreamAlignedBatcher – builds batches with a fixed number of examples per class, grouped together, required for I_avg's per-class averaging (Eq. 7 of "Faster Predictive Coding Networks via Better Initialization") |
| Timer.h | |
| src | |
| backend | |
| Backend.cpp | |
| CPUBackend.cpp | |
| Tensor.cpp | |
| layers | |
| ConvPCLayer.cpp | |
| DirectKPPCLayer.cpp | |
| DiscriminativePCLayer.cpp | |
| FullPCLayer.cpp | |
| GaussSeidelPCLayer.cpp | |
| RBLayer.cpp | |
| SimpleConvPCLayer.cpp | |
| SimplePCLayer.cpp | |
| networks | |
| ConvPCNetwork.cpp | |
| DirectKPPCNetwork.cpp | |
| DiscriminativePCNetwork.cpp | |
| FullPCNetwork.cpp | |
| GaussSeidelPCNetwork.cpp | |
| SimpleConvPCNetwork.cpp | |
| SimplePCNetwork.cpp | |
| utils | |
| StreamAlignedBatcher.cpp |
1.16.1