FullPCLayer.h File Reference

New, standalone PC layer (not derived from DirectKPPCLayer) meant to eventually carry every technique from the paper-combination work – muPC scaling, optional residual connections, DKP direct feedback, AdamW, muP-style per-layer LR scaling, optional iPC, cross-entropy terminal loss – each one OFF by default, opt-in per layer/network, so a plain DKP-PC-equivalent network is just this class with every extra left at its default. More...

#include <deepity/backend/IComputeBackend.h>
#include <deepity/layers/Layer.h>
#include <deepity/utils/Activations.h>
#include <deepity/utils/AdamOptimizer.h>
#include <deepity/utils/DeviceMemoryArena.h>
#include <deepity/utils/MemoryArena.h>
#include <map>
#include <memory>
#include <random>
#include <vector>

Go to the source code of this file.

Classes
class	Deep::FullPCLayer
	PC layer combining muPC scaling, optional residual connections, and DKP direct feedback – every extra OFF by default, so plain defaults reproduce DirectKPPCLayer exactly. See the file-level note above for the full design rationale. More...

Namespaces
namespace	Deep
	This file exists to provide both Tensor.h and IComputeBackend.h with device types without introducing circular dependencies.

Detailed Description

New, standalone PC layer (not derived from DirectKPPCLayer) meant to eventually carry every technique from the paper-combination work – muPC scaling, optional residual connections, DKP direct feedback, AdamW, muP-style per-layer LR scaling, optional iPC, cross-entropy terminal loss – each one OFF by default, opt-in per layer/network, so a plain DKP-PC-equivalent network is just this class with every extra left at its default.

THIS PASS: only muPC scaling (a) and residual connections (useResidual) are added, on top of DirectKPPCLayer's proven structure (DKP feedback, Adam/AdamW, graph-capture-compatible device-pointer t/lr). Everything else stays for a later pass.

a and useResidual are set by the owning network at Compile() time (via SetMuPCScale/SetResidual), not computed here – this layer, at construction time, doesn't know the full network's shape (width N, depth L, input dim) that muPC's own aL formula needs. Defaults (a=1.0, useResidual=false) reproduce plain, unscaled PC exactly, matching the "every extra opt-in, off by default" design.