A PC layer whose settling dynamics follow a Gauss-Seidel (sequential-sweep) update, matching ngc-learn's actual execution order – NOT the Jacobi (fully-synchronous) update SimplePCLayer uses. More...

#include <vector>
#include <stdexcept>
#include <random>
#include <memory>
#include <cstdlib>
#include <deepity/utils/Activations.h>
#include <deepity/utils/AdamOptimizer.h>
#include <deepity/layers/Layer.h>
#include <deepity/utils/MemoryArena.h>

Classes
class	Deep::GaussSeidelPCLayer
	PC layer with Gauss-Seidel (sequential-sweep) settling dynamics, in place of the usual simultaneous update. Not yet gradient-checked – see the file-level warning above. More...

Namespaces
namespace	Deep
	This file exists to provide both Tensor.h and IComputeBackend.h with device types without introducing circular dependencies.

Detailed Description

A PC layer whose settling dynamics follow a Gauss-Seidel (sequential-sweep) update, matching ngc-learn's actual execution order – NOT the Jacobi (fully-synchronous) update SimplePCLayer uses.

Note: THIS IS AN UNPROVEN EXPERIMENT, not a confirmed improvement. Classical numerical-methods results suggest Gauss-Seidel typically needs fewer iterations than Jacobi for diagonally-dominant/symmetric systems, and our CPU/OpenMP implementation was never exploiting cross-layer parallelism anyway (unlike a GPU/JAX backend, where Gauss-Seidel's sequential dependency chain is a real cost) – so the downside risk looks smaller here than it would for a GPU implementation. But this has NOT been empirically confirmed for this specific nonlinear, learned-weight system. Gradient-check independently before trusting it, same discipline as every other new formula this session.

KEY STRUCTURAL DIFFERENCE from SimplePCLayer: there is no single CalculateState() that does everything. Traced directly from ngc-learn's real advance_process chain (E2,E3 -> z0,z1,z2,z3 -> W1,W2, W3 -> e1,e2,e3), a full Gauss-Seidel timestep is THREE separate sweeps across ALL layers, in this exact order:

UpdateState() on every layer – z updates using mu/e_above HELD OVER from the end of the PREVIOUS timestep (not yet fresh for this timestep).
ComputePrediction() on every layer – mu recomputes fresh, using the JUST-updated z from step 1. Order among layers doesn't matter here (each layer's mu only depends on its OWN z).
ComputeError() on every layer – e recomputes fresh, using the JUST-updated z (step 1) as the target and layerBelow's FRESH mu (step 2) as the prediction. Order among layers doesn't matter here either.

This differs from SimplePCLayer's single-call CalculateState(), which computes error and prediction together, and whose settling loop is simply (CalculateState(); UpdateState();) x N – fully synchronous (Jacobi): every layer only ever reads values from the END of the PREVIOUS full step, never a value updated earlier in the SAME step.

A GaussSeidelPCNetwork class is required to actually orchestrate the three sweeps above in the correct order across all layers – calling these methods directly, out of order, or on a single layer in isolation, will not reproduce the intended dynamics.

IMPORTANT BUFFER-LIFETIME NOTE (the same class of bug mu-caching hit earlier): mu must NOT be mutated in place into its own derivative the way SimplePCLayer's UpdateState() does – it needs to stay in its clean, activated form, since the layer ABOVE's ComputeError() reads it later in this SAME timestep. UpdateState() and UpdateWeights() each compute mu's derivative into a separate scratch buffer (muDeriv) instead, leaving mu itself untouched.

Deliberately does NOT include mu-caching or activateBeforeTransform in this first version – isolating the Gauss-Seidel restructuring itself for independent verification before layering anything else on top.

Warning: Not yet gradient-checked. Do that before trusting this for real training.

Version: 1.0

Date: 2026-08-26

Classes

Namespaces

Detailed Description