Deep::RBLayer Class Reference

Restricted Boltzmann-style Predictive Coding (PC) Layer. More...

#include <RBLayer.h>

Inheritance diagram for Deep::RBLayer:

Public Member Functions
	RBLayer (size_t inSize, size_t outSize, float var=1.0f, float var_td=10.0f, float k_1=1e-3f, float k_2=1e-5f, float lmbda=1e-6f, float alpha=1.0f, size_t batchSize=64, int stepSize=30, void(*act)(float *, size_t)=relu, void(*dAct)(float *, size_t, bool)=dRelu)
	Constructs an RBLayer with specified dimensions and hyperparameters.
	~RBLayer () override
void	RunPrediction (const float *input, size_t currentBatchSize) noexcept
	Executes a forward pass prediction without updating internal beliefs.
void	RunInferenceStep (const float *input, const float *topDown, size_t currentBatchSize) noexcept
	Executes a single bidirectional inference step integrating bottom-up and top-down signals.
void	CalcError (const float *bottomUp, const float *topDown, size_t currentBatchSize) noexcept
	Computes the precision-weighted prediction errors.
void	UpdateBeliefs (const float *bottomUp, const float *topDown, size_t currentBatchSize) noexcept
	Updates internal state beliefs via gradient descent on the prediction error.
void	UpdateWeights (size_t currentBatchSize) noexcept
	Applies batched gradients to update the synaptic weights.
float	CalculateState () noexcept override
	Calculates the total layer energy state.
void	UpdateState () noexcept override
	Computes the state derivatives and applies them to current beliefs.
void	UpdateWeights () noexcept override
	Parameter-less override for generic layer weight updates.
void	Flush () noexcept override
	Flushes remaining batches.
const float *	GetErrors () const noexcept override
	Should return errors e.
size_t	GetTotalSize () const noexcept
	Returns the total byte size required by this layer for flat allocation.
void	Attach (float *ptr) noexcept
	Attaches the layer to a pre-allocated, 64-byte aligned memory block.
float *	GetWeights () const noexcept
	Returns the synaptic weight matrix U.
size_t	GetBatchSize () const noexcept override
	Should return batch size.
size_t	GetInputSize () const noexcept override
	Should return input size.
size_t	GetOutputSize () const noexcept override
	Should return output size.
float *	GetBeliefs () noexcept override
	Should return belief state z.
const float *	GetInferenceError () const noexcept
	Same underlying buffer as GetErrors() – the bottom-up prediction error, named separately here for call sites that think in terms of RBLayer's own inference-error terminology.
Public Member Functions inherited from Deep::Layer
virtual	~Layer ()=default
virtual std::map< std::string, TensorDescriptor >	GetStateDict () const
	Returns a dictionary of this.

Additional Inherited Members
Protected Attributes inherited from Deep::Layer
size_t	size
	Size of input.
size_t	nextSize
	Size of output.

Detailed Description

Restricted Boltzmann-style Predictive Coding (PC) Layer.

See also

https://www.nature.com/articles/nn0199_79.pdf

Constructor & Destructor Documentation

RBLayer()

RBLayer::RBLayer	(	size_t	inSize,
		size_t	outSize,
		float	var = 1.0f,
		float	var_td = 10.0f,
		float	k_1 = 1e-3f,
		float	k_2 = 1e-5f,
		float	lmbda = 1e-6f,
		float	alpha = 1.0f,
		size_t	batchSize = 64,
		int	stepSize = 30,
		void(*	act )(float *, size_t) = relu,
		void(*	dAct )(float *, size_t, bool) = dRelu )

Constructs an RBLayer with specified dimensions and hyperparameters.

• Parameters

  inSize	Dimension of the bottom-up input.
  outSize	Dimension of the top-down output (layer state size).
  var	Bottom-up variance (precision inverse).
  var_td	Top-down variance (precision inverse).
  k_1	Forward integration time constant.
  k_2	Backward integration time constant.
  lmbda	Weight decay (L2 regularization) coefficient.
  alpha	Learning rate for weight updates.
  batchSize	Number of parallel sequences to process.
  stepSize	Number of inference iterations per state update.
  act	Forward activation function (SIMD optimized).
  dAct	Derivative of the activation function (SIMD optimized).

~RBLayer()

RBLayer::~RBLayer ( )

override

Member Function Documentation

Attach()

void RBLayer::Attach ( float * ptr )

noexcept

Attaches the layer to a pre-allocated, 64-byte aligned memory block.

CalcError()

void RBLayer::CalcError ( const float * bottomUp, const float * topDown, size_t currentBatchSize )

noexcept

Computes the precision-weighted prediction errors.

CalculateState()

float RBLayer::CalculateState ( )

nodiscardoverridevirtualnoexcept

Calculates the total layer energy state.

Computes the sum of squared bottom-up prediction errors:

\[E = \frac{1}{2} ||e_{bu}||^2 = \frac{1}{2} \sum (I - rU)^2 \]

Implements Deep::Layer.

Flush()

void RBLayer::Flush ( )

overridevirtualnoexcept

Flushes remaining batches.

Reimplemented from Deep::Layer.

GetBatchSize()

size_t RBLayer::GetBatchSize ( ) const

nodiscardoverridevirtualnoexcept

Should return batch size.

Returns

? = 0

Implements Deep::Layer.

GetBeliefs()

float * RBLayer::GetBeliefs ( )

nodiscardoverridevirtualnoexcept

Should return belief state z.

Returns

? = 0

Implements Deep::Layer.

GetErrors()

const float * Deep::RBLayer::GetErrors ( ) const

inlinenodiscardoverridevirtualnoexcept

Should return errors e.

Returns

? = 0

Implements Deep::Layer.

GetInferenceError()

const float * RBLayer::GetInferenceError ( ) const

nodiscardnoexcept

Same underlying buffer as GetErrors() – the bottom-up prediction error, named separately here for call sites that think in terms of RBLayer's own inference-error terminology.

GetInputSize()

size_t RBLayer::GetInputSize ( ) const

nodiscardoverridevirtualnoexcept

Should return input size.

Returns

? = 0

Implements Deep::Layer.

GetOutputSize()

size_t RBLayer::GetOutputSize ( ) const

nodiscardoverridevirtualnoexcept

Should return output size.

Returns

? = 0

Implements Deep::Layer.

GetTotalSize()

size_t RBLayer::GetTotalSize ( ) const

nodiscardnoexcept

Returns the total byte size required by this layer for flat allocation.

GetWeights()

float * Deep::RBLayer::GetWeights ( ) const

inlinenodiscardnoexcept

Returns the synaptic weight matrix U.

RunInferenceStep()

void RBLayer::RunInferenceStep ( const float * input, const float * topDown, size_t currentBatchSize )

noexcept

Executes a single bidirectional inference step integrating bottom-up and top-down signals.

RunPrediction()

void RBLayer::RunPrediction ( const float * input, size_t currentBatchSize )

noexcept

Executes a forward pass prediction without updating internal beliefs.

UpdateBeliefs()

void RBLayer::UpdateBeliefs ( const float * bottomUp, const float * topDown, size_t currentBatchSize )

noexcept

Updates internal state beliefs via gradient descent on the prediction error.

Integrates the state dynamics using the forward error, backward error, and a Gaussian prior (decay):

\[r \leftarrow r + k_1 \left( \frac{1}{\sigma^2} (I - rU)U^T + \frac{1}{\sigma^2_{td}}(td - r) - \alpha r \right) \]

Parameters

bottomUp	The input data \( I \) from the layer below.
topDown	The target data \( td \) from the layer above (can be nullptr).
currentBatchSize	The active batch dimension.

UpdateState()

void RBLayer::UpdateState ( )

overridevirtualnoexcept

Computes the state derivatives and applies them to current beliefs.

Wrapper for UpdateBeliefs() using the internally cached input batch.

\[r \leftarrow r + k_1 \left( \frac{1}{\sigma^2} (I - rU)U^T - \alpha r \right) \]

Implements Deep::Layer.

UpdateWeights() [1/2]

void RBLayer::UpdateWeights ( )

overridevirtualnoexcept

Parameter-less override for generic layer weight updates.

Wrapper for the batched UpdateWeights() using the internally cached input batch.

\[\Delta U = \frac{k_2}{\sigma^2} r^T \left( (I - f(rU)) \odot f'(rU) \right) - k_2 \lambda U \]

Implements Deep::Layer.

UpdateWeights() [2/2]

void RBLayer::UpdateWeights ( size_t currentBatchSize )

noexcept

Applies batched gradients to update the synaptic weights.

Computes the weight update using the non-linear prediction error, applies L2 weight decay, and projects the rows of the weight matrix back to a unit norm:

\[\Delta U = \frac{k_2}{\sigma^2} r^T \left( (I - f(rU)) \odot f'(rU) \right) - k_2 \lambda U \]

Parameters

currentBatchSize

The active batch dimension.

---

The documentation for this class was generated from the following files:

• include/deepity/layers/RBLayer.h
• src/layers/RBLayer.cpp