Device-agnostic compute backend interface: every numerical primitive a layer/network needs, implemented once each by CPUBackend (SIMD/BLAS) and CUDABackend (kernels/cuBLAS/CUTLASS), so layer code never branches on device directly. More...

#include <IComputeBackend.h>

Inheritance diagram for Deep::IComputeBackend:

Public Member Functions
virtual	~IComputeBackend ()=default
virtual void	BeginGraphCapture () noexcept=0
	Starts recording subsequent backend calls into a replayable graph, instead of executing them immediately (CUDA graph capture; a no-op record on CPUBackend). Pair with EndGraphCapture().
virtual bool	EndGraphCapture () noexcept=0
	Ends capture and instantiates the captured graph.
virtual void	ReplayGraph () noexcept=0
	Launches a graph previously captured and instantiated by BeginGraphCapture()/EndGraphCapture(). No-op if EndGraphCapture() never returned true.
virtual float *	Allocate (size_t numFloats)=0
	Allocates numFloats floats on this backend's device.
virtual void	Free (float *ptr) noexcept=0
	Frees a buffer previously returned by Allocate(). No-op on nullptr.
virtual void	Zero (float *ptr, size_t numFloats) noexcept=0
	Zero-fills numFloats floats starting at ptr.
virtual void	Copy (float dst, const float src, size_t numFloats) noexcept=0
	Copies numFloats floats from src to dst, both on this backend's own device.
virtual void	CopyFromHost (float deviceDst, const float hostSrc, size_t numFloats) noexcept=0
	Copies numFloats floats from a host buffer into a device buffer (a plain memcpy on CPUBackend).
virtual void	CopyToHost (float hostDst, const float deviceSrc, size_t numFloats) noexcept=0
	Copies numFloats floats from a device buffer into a host buffer (a plain memcpy on CPUBackend).
virtual void	RandomizeNormal (float *buf, size_t n, float mean, float stddev, uint32_t seed) noexcept=0
	Fills n floats with samples from a normal distribution.
virtual void	RandomizeUniform (float *buf, size_t n, float min, float max, uint32_t seed) noexcept=0
	Fills n floats with samples from a uniform distribution over [min, max).
virtual void	PrepareForBatchSize (size_t batchSize) noexcept=0
	Must be called once, before Compile()'s first BeginGraphCapture(), for any backend that needs to prepare batch-size-dependent state (e.g. CUDABackend's cached all-ones vector for SumRows' GEMV). No-op on CPUBackend.
virtual void	MatMul (bool transA, bool transB, int M, int N, int K, float alpha, const float A, int lda, const float B, int ldb, float beta, float *C, int ldc) noexcept=0
	C = alpha * op(A) * op(B) + beta * C, the standard BLAS SGEMM convention (op(X) = X^T if the matching transX is true).
virtual void	SumRows (float dst, const float src, size_t batchSize, size_t width) noexcept=0
	dst[j] = sum over b in [0,batchSize) of src[b*width + j], for all j in [0,width). Replaces a batchSize-iteration loop of individual AxpyInto calls – the reduction-direction counterpart to AddBiasBroadcast, still unfixed until now. On GPU this is one cublasSgemv call against a cached all-ones vector, reinterpreting src's row-major [batchSize,width] layout as column-major [width,batchSize] with no data movement (verified numerically).
virtual void	Scale (float *buf, size_t n, float alpha) noexcept=0
	buf[i] *= alpha for all i in [0, n).
virtual void	AxpyInto (float y, const float x, size_t n, float alpha) noexcept=0
	y[i] += alpha * x[i] for all i in [0, n) – the standard AXPY.
virtual void	AddBiasBroadcast (float buf, const float bias, size_t batchSize, size_t width) noexcept=0
	buf[row*width + col] += bias[col] for every row in [0,batchSize) and col in [0,width) – broadcasts a per-column bias across every row (the dense-layer convention).
virtual void	Activation (ActivationType type, float *buf, size_t n) noexcept=0
	Applies type to buf in place.
virtual void	ActivationInto (ActivationType type, float dst, const float src, size_t n) noexcept=0
	Two-buffer variant of Activation(): reads src, writes into dst.
virtual void	ActivationDerivative (ActivationType type, float *buf, size_t n, bool activated) noexcept=0
	Applies type's derivative to buf in place.
virtual void	ActivationDerivativeInto (ActivationType type, float dst, const float src, size_t n) noexcept=0
	Two-buffer variant of ActivationDerivative(): reads src (the pre-activation value), writes the derivative into dst.
virtual void	FusedStateUpdateMomentum (float z, float v, const float feedback, const float deriv, const float *e, size_t n, float ir, float beta) noexcept=0
	Same as FusedStateUpdate, but with momentum: the update direction (feedbackderiv - e) is EMA-smoothed into v (same shape as z, persistent across settling steps within one TrainStep(), reset to zero at the start of each call) before being applied to z. beta is the EMA decay (0.9 is a common default – higher = more smoothing/inertia). v = betav + (1-beta)((feedbackderiv) - e) z += ir*v.
virtual void	FusedStateUpdate (float z, const float feedback, const float deriv, const float e, size_t n, float ir) noexcept=0
	One settling step's state update, fused into a single call: z += ir * ((feedback * deriv) - e) Non-momentum counterpart to FusedStateUpdateMomentum() above.
virtual float	ComputeErrorAndEnergy (float e, const float z, const float *mu, size_t n) noexcept=0
	Computes e = z - mu, then returns 0.5 * sum(e^2) – the Gaussian error/energy used by every layer except a cross-entropy terminal.
virtual void	ComputeError (float e, const float z, const float *mu, size_t n) noexcept=0
	Same as ComputeErrorAndEnergy(), without computing (or returning) the energy – for callers that only need e.
virtual float	ComputeSoftmaxCrossEntropyErrorAndEnergy (float e, const float z, const float mu, size_t batchSize, size_t nextSize, float rowEnergies) noexcept=0
	Softmax cross-entropy variant of ComputeErrorAndEnergy, for the terminal layer only. Unlike the Gaussian version, this needs batchSize/nextSize separately (not just a flat n) since softmax is a row-wise, not element-wise, operation. probs = softmax(mu), row-wise e = z - probs (z is the clamped target; matches the existing e := -dF/dmu convention exactly, since d(CE)/d(logits) = probs - y) energy = -sum(z * log(probs + eps)) (eps=1e-8, avoids log(0)) mu holds the RAW LOGITS (unchanged from the Gaussian case, still a*W@phi(z)+b) – softmax is applied here, not baked into mu itself. rowEnergies is caller-provided scratch, >= batchSize floats – NOT allocated internally (this runs inside CalculateState(), which runs inside TrainStep()'s CUDA-graph-captured region; dynamic allocation there is unsafe under capture, same reasoning as the CUTLASS workspace). CPUBackend's own override ignores this parameter entirely – kept in the shared interface so callers pass the same arguments to either backend.
virtual void	ComputeSoftmaxCrossEntropyError (float e, const float z, const float *mu, size_t batchSize, size_t nextSize) noexcept=0
	Same as ComputeSoftmaxCrossEntropyErrorAndEnergy(), without computing (or requiring rowEnergies for) the energy – for callers that only need e.
virtual bool	TryFusedForwardPass (ActivationType actType, const float zF, const float W, const float bias, float mu, int batchSize, int size, int nextSize) noexcept=0
	Attempts a fused forward pass (GEMM + bias + activation in one kernel, via CUTLASS on GPU) for RELU or LINEAR activation types only – see CUDABackend's implementation for why other activation types aren't attempted yet. Returns true if the fused path was used (mu is fully computed, including bias and activation); false if the caller should fall back to the existing MatMul+AddBiasBroadcast+ Activation sequence (always false on CPUBackend – CPU has no fused path, this is a GPU-only optimization).
virtual void	Im2Col (const float input, int channels, int height, int width, int kernelH, int kernelW, int strideH, int strideW, int padH, int padW, float columns) noexcept=0
	Rearranges a single (channels, height, width) input image into a (channelskHkW, outH*outW) column matrix – the standard im2col transform. NOT batch-aware: call once per batch item, with `input` and `columns` offset to that item's slice, matching Deep::Im2Col's own documented contract exactly (CPUBackend forwards to it directly). Positions outside the input (due to padding) are written as zero. `columns` is fully overwritten, not accumulated into.
virtual void	Col2Im (const float columns, int channels, int height, int width, int kernelH, int kernelW, int strideH, int strideW, int padH, int padW, float outputImage) noexcept=0
	The adjoint of Im2Col(): scatters a (channelskHkW, outH*outW) column-gradient buffer back into a (channels, height, width) image. NOT batch-aware, same per-item-offset contract as Im2Col(). ACCUMULATES into `outputImage` (does not zero it first) – caller must zero the destination if a fresh result is wanted, matching Deep::Col2Im's own contract exactly.
virtual void	RepackForBatchedGemm (float dst, const float src, size_t batchSize, size_t rows, size_t cols) noexcept=0
	Repacks a [batchSize, rows, cols] tensor (batch-major) into [rows, batchSize, cols] (row-major, batch second) – i.e. dst[row][batch][col] = src[batch][row][col] for all row/batch/col, with the innermost cols dimension kept contiguous on both sides. Used by ConvPCLayer-family layers to reorganize per-batch im2col columns (and per-batch upstream-error columns) into the single flat layout a plain (non-batched) GEMM call needs, rather than one GEMM call per batch item. Same "port first, optimize" position as AddBiasBroadcast/SumRows: a future strided-batched-GEMM path could remove the need for this entirely, but this matches the existing, verified CPU repacking loops exactly for now.
virtual void	IncrementCounter (int *counter) noexcept=0
	Atomically increments *counter by 1. Device-resident so it can be read/updated by the same kernel/graph across settling steps (e.g. Adam's step count t) without a host round-trip.
virtual void	AdamStep (float param, const float grad, float m, float v, size_t n, const int t, const float lr, float beta1=0.9f, float beta2=0.999f, float eps=1e-8f) noexcept=0
	Adam update for n parameters in place.
virtual void	AdamWStep (float param, const float grad, float m, float v, size_t n, const int t, const float lr, float weightDecay, float beta1=0.9f, float beta2=0.999f, float eps=1e-8f) noexcept=0
	Same as AdamStep(), plus decoupled weight decay (AdamW): param -= lr * weightDecay * param, applied before the Adam step itself.
virtual DeviceType	GetDeviceType () const noexcept=0
	Which device this backend instance runs on.
virtual void	MultiplyInto (float dst, const float a, const float *b, size_t n) noexcept=0
	dst[i] = a[i] * b[i] for all i in [0, n) – elementwise product.
virtual void	Fill (float *buf, size_t n, float value) noexcept=0
	buf[i] = value for all i in [0, n).
virtual void	AddBiasPerChannel (float buf, const float bias, size_t channels, size_t spatialSize) noexcept=0
	Convolutional bias-add: buf[c*spatialSize + s] += bias[c] for all c in [0,channels), s in [0,spatialSize). Per-CHANNEL broadcast across spatial positions – the transpose relationship to AddBiasBroadcast (which broadcasts a per-COLUMN bias across ROWS, the dense-layer convention). NOT batch-aware: matches Im2Col/ Col2Im's contract exactly – call once per batch item, with `buf` offset to that item's slice.

Detailed Description

Device-agnostic compute backend interface: every numerical primitive a layer/network needs, implemented once each by CPUBackend (SIMD/BLAS) and CUDABackend (kernels/cuBLAS/CUTLASS), so layer code never branches on device directly.

Constructor & Destructor Documentation

◆ ~IComputeBackend()

virtual Deep::IComputeBackend::~IComputeBackend ( )

virtualdefault

Member Function Documentation

◆ Activation()

virtual void Deep::IComputeBackend::Activation	(	ActivationType	type,
		float *	buf,
		size_t	n )

pure virtualnoexcept

Applies type to buf in place.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ActivationDerivative()

virtual void Deep::IComputeBackend::ActivationDerivative	(	ActivationType	type,
		float *	buf,
		size_t	n,
		bool	activated )

pure virtualnoexcept

Applies type's derivative to buf in place.

Parameters

type	Which activation's derivative to apply.
buf	Array to derive in place.
n	Length of buf.
activated	If true, buf already holds the activated value; if false, buf holds the pre-activation value.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ActivationDerivativeInto()

virtual void Deep::IComputeBackend::ActivationDerivativeInto	(	ActivationType	type,
		float *	dst,
		const float *	src,
		size_t	n )

pure virtualnoexcept

Two-buffer variant of ActivationDerivative(): reads src (the pre-activation value), writes the derivative into dst.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ActivationInto()

virtual void Deep::IComputeBackend::ActivationInto	(	ActivationType	type,
		float *	dst,
		const float *	src,
		size_t	n )

pure virtualnoexcept

Two-buffer variant of Activation(): reads src, writes into dst.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ AdamStep()

virtual void Deep::IComputeBackend::AdamStep	(	float *	param,
		const float *	grad,
		float *	m,
		float *	v,
		size_t	n,
		const int *	t,
		const float *	lr,
		float	beta1 = 0.9f,
		float	beta2 = 0.999f,
		float	eps = 1e-8f )

pure virtualnoexcept

Adam update for n parameters in place.

Parameters

param	Parameters to update.
grad	Gradient w.r.t. param.
m	Adam's first-moment (momentum) buffer, persistent across calls.
v	Adam's second-moment (variance) buffer, persistent across calls.
n	Number of parameters.
t	Device-resident step count (read, not written, here – see IncrementCounter()).
lr	Device-resident learning rate.
beta1,beta2	Momentum/variance EMA decay rates.
eps	Denominator stabilizer.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ AdamWStep()

virtual void Deep::IComputeBackend::AdamWStep	(	float *	param,
		const float *	grad,
		float *	m,
		float *	v,
		size_t	n,
		const int *	t,
		const float *	lr,
		float	weightDecay,
		float	beta1 = 0.9f,
		float	beta2 = 0.999f,
		float	eps = 1e-8f )

pure virtualnoexcept

Same as AdamStep(), plus decoupled weight decay (AdamW): param -= lr * weightDecay * param, applied before the Adam step itself.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ AddBiasBroadcast()

virtual void Deep::IComputeBackend::AddBiasBroadcast	(	float *	buf,
		const float *	bias,
		size_t	batchSize,
		size_t	width )

pure virtualnoexcept

buf[row*width + col] += bias[col] for every row in [0,batchSize) and col in [0,width) – broadcasts a per-column bias across every row (the dense-layer convention).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ AddBiasPerChannel()

virtual void Deep::IComputeBackend::AddBiasPerChannel	(	float *	buf,
		const float *	bias,
		size_t	channels,
		size_t	spatialSize )

pure virtualnoexcept

Convolutional bias-add: buf[c*spatialSize + s] += bias[c] for all c in [0,channels), s in [0,spatialSize). Per-CHANNEL broadcast across spatial positions – the transpose relationship to AddBiasBroadcast (which broadcasts a per-COLUMN bias across ROWS, the dense-layer convention). NOT batch-aware: matches Im2Col/ Col2Im's contract exactly – call once per batch item, with buf offset to that item's slice.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Allocate()

virtual float * Deep::IComputeBackend::Allocate ( size_t numFloats )

pure virtual

Allocates numFloats floats on this backend's device.

Returns: nullptr on allocation failure.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ AxpyInto()

virtual void Deep::IComputeBackend::AxpyInto	(	float *	y,
		const float *	x,
		size_t	n,
		float	alpha )

pure virtualnoexcept

y[i] += alpha * x[i] for all i in [0, n) – the standard AXPY.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ BeginGraphCapture()

virtual void Deep::IComputeBackend::BeginGraphCapture ( )

pure virtualnoexcept

Starts recording subsequent backend calls into a replayable graph, instead of executing them immediately (CUDA graph capture; a no-op record on CPUBackend). Pair with EndGraphCapture().

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Col2Im()

virtual void Deep::IComputeBackend::Col2Im	(	const float *	columns,
		int	channels,
		int	height,
		int	width,
		int	kernelH,
		int	kernelW,
		int	strideH,
		int	strideW,
		int	padH,
		int	padW,
		float *	outputImage )

pure virtualnoexcept

The adjoint of Im2Col(): scatters a (channels*kH*kW, outH*outW) column-gradient buffer back into a (channels, height, width) image. NOT batch-aware, same per-item-offset contract as Im2Col(). ACCUMULATES into outputImage (does not zero it first) – caller must zero the destination if a fresh result is wanted, matching Deep::Col2Im's own contract exactly.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ComputeError()

virtual void Deep::IComputeBackend::ComputeError	(	float *	e,
		const float *	z,
		const float *	mu,
		size_t	n )

pure virtualnoexcept

Same as ComputeErrorAndEnergy(), without computing (or returning) the energy – for callers that only need e.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ComputeErrorAndEnergy()

virtual float Deep::IComputeBackend::ComputeErrorAndEnergy	(	float *	e,
		const float *	z,
		const float *	mu,
		size_t	n )

pure virtualnoexcept

Computes e = z - mu, then returns 0.5 * sum(e^2) – the Gaussian error/energy used by every layer except a cross-entropy terminal.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ComputeSoftmaxCrossEntropyError()

virtual void Deep::IComputeBackend::ComputeSoftmaxCrossEntropyError	(	float *	e,
		const float *	z,
		const float *	mu,
		size_t	batchSize,
		size_t	nextSize )

pure virtualnoexcept

Same as ComputeSoftmaxCrossEntropyErrorAndEnergy(), without computing (or requiring rowEnergies for) the energy – for callers that only need e.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ComputeSoftmaxCrossEntropyErrorAndEnergy()

virtual float Deep::IComputeBackend::ComputeSoftmaxCrossEntropyErrorAndEnergy	(	float *	e,
		const float *	z,
		const float *	mu,
		size_t	batchSize,
		size_t	nextSize,
		float *	rowEnergies )

pure virtualnoexcept

Softmax cross-entropy variant of ComputeErrorAndEnergy, for the terminal layer only. Unlike the Gaussian version, this needs batchSize/nextSize separately (not just a flat n) since softmax is a row-wise, not element-wise, operation. probs = softmax(mu), row-wise e = z - probs (z is the clamped target; matches the existing e := -dF/dmu convention exactly, since d(CE)/d(logits) = probs - y) energy = -sum(z * log(probs + eps)) (eps=1e-8, avoids log(0)) mu holds the RAW LOGITS (unchanged from the Gaussian case, still a*W@phi(z)+b) – softmax is applied here, not baked into mu itself. rowEnergies is caller-provided scratch, >= batchSize floats – NOT allocated internally (this runs inside CalculateState(), which runs inside TrainStep()'s CUDA-graph-captured region; dynamic allocation there is unsafe under capture, same reasoning as the CUTLASS workspace). CPUBackend's own override ignores this parameter entirely – kept in the shared interface so callers pass the same arguments to either backend.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Copy()

virtual void Deep::IComputeBackend::Copy	(	float *	dst,
		const float *	src,
		size_t	numFloats )

pure virtualnoexcept

Copies numFloats floats from src to dst, both on this backend's own device.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ CopyFromHost()

virtual void Deep::IComputeBackend::CopyFromHost	(	float *	deviceDst,
		const float *	hostSrc,
		size_t	numFloats )

pure virtualnoexcept

Copies numFloats floats from a host buffer into a device buffer (a plain memcpy on CPUBackend).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ CopyToHost()

virtual void Deep::IComputeBackend::CopyToHost	(	float *	hostDst,
		const float *	deviceSrc,
		size_t	numFloats )

pure virtualnoexcept

Copies numFloats floats from a device buffer into a host buffer (a plain memcpy on CPUBackend).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ EndGraphCapture()

virtual bool Deep::IComputeBackend::EndGraphCapture ( )

pure virtualnoexcept

Ends capture and instantiates the captured graph.

Returns: true if capture and instantiation both succeeded and ReplayGraph() is now safe to call; false otherwise. Callers MUST check this – silently assuming success here was the cause of a real bug: a failed capture left ReplayGraph() permanently doing nothing on every subsequent call, since the caller had no way to know capture never actually happened.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Fill()

virtual void Deep::IComputeBackend::Fill	(	float *	buf,
		size_t	n,
		float	value )

pure virtualnoexcept

buf[i] = value for all i in [0, n).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Free()

virtual void Deep::IComputeBackend::Free ( float * ptr )

pure virtualnoexcept

Frees a buffer previously returned by Allocate(). No-op on nullptr.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ FusedStateUpdate()

virtual void Deep::IComputeBackend::FusedStateUpdate	(	float *	z,
		const float *	feedback,
		const float *	deriv,
		const float *	e,
		size_t	n,
		float	ir )

pure virtualnoexcept

One settling step's state update, fused into a single call: z += ir * ((feedback * deriv) - e) Non-momentum counterpart to FusedStateUpdateMomentum() above.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ FusedStateUpdateMomentum()

virtual void Deep::IComputeBackend::FusedStateUpdateMomentum	(	float *	z,
		float *	v,
		const float *	feedback,
		const float *	deriv,
		const float *	e,
		size_t	n,
		float	ir,
		float	beta )

pure virtualnoexcept

Same as FusedStateUpdate, but with momentum: the update direction (feedback*deriv - e) is EMA-smoothed into v (same shape as z, persistent across settling steps within one TrainStep(), reset to zero at the start of each call) before being applied to z. beta is the EMA decay (0.9 is a common default – higher = more smoothing/inertia). v = beta*v + (1-beta)*((feedback*deriv) - e) z += ir*v.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ GetDeviceType()

virtual DeviceType Deep::IComputeBackend::GetDeviceType ( ) const

pure virtualnoexcept

Which device this backend instance runs on.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Im2Col()

virtual void Deep::IComputeBackend::Im2Col	(	const float *	input,
		int	channels,
		int	height,
		int	width,
		int	kernelH,
		int	kernelW,
		int	strideH,
		int	strideW,
		int	padH,
		int	padW,
		float *	columns )

pure virtualnoexcept

Rearranges a single (channels, height, width) input image into a (channels*kH*kW, outH*outW) column matrix – the standard im2col transform. NOT batch-aware: call once per batch item, with input and columns offset to that item's slice, matching Deep::Im2Col's own documented contract exactly (CPUBackend forwards to it directly). Positions outside the input (due to padding) are written as zero. columns is fully overwritten, not accumulated into.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ IncrementCounter()

virtual void Deep::IComputeBackend::IncrementCounter ( int * counter )

pure virtualnoexcept

Atomically increments *counter by 1. Device-resident so it can be read/updated by the same kernel/graph across settling steps (e.g. Adam's step count t) without a host round-trip.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ MatMul()

virtual void Deep::IComputeBackend::MatMul	(	bool	transA,
		bool	transB,
		int	M,
		int	N,
		int	K,
		float	alpha,
		const float *	A,
		int	lda,
		const float *	B,
		int	ldb,
		float	beta,
		float *	C,
		int	ldc )

pure virtualnoexcept

C = alpha * op(A) * op(B) + beta * C, the standard BLAS SGEMM convention (op(X) = X^T if the matching transX is true).

Parameters

transA	Whether to transpose A.
transB	Whether to transpose B.
M	Rows of op(A) and C.
N	Columns of op(B) and C.
K	Columns of op(A), rows of op(B).
alpha	Scalar multiplied into op(A)*op(B).
A	Left operand.
lda,ldb,ldc	Leading dimensions of A, B, C respectively.
B	Right operand.
beta	Scalar multiplied into the existing C before accumulating.
C	Output; also read if beta != 0.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ MultiplyInto()

virtual void Deep::IComputeBackend::MultiplyInto	(	float *	dst,
		const float *	a,
		const float *	b,
		size_t	n )

pure virtualnoexcept

dst[i] = a[i] * b[i] for all i in [0, n) – elementwise product.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ PrepareForBatchSize()

virtual void Deep::IComputeBackend::PrepareForBatchSize ( size_t batchSize )

pure virtualnoexcept

Must be called once, before Compile()'s first BeginGraphCapture(), for any backend that needs to prepare batch-size-dependent state (e.g. CUDABackend's cached all-ones vector for SumRows' GEMV). No-op on CPUBackend.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ RandomizeNormal()

virtual void Deep::IComputeBackend::RandomizeNormal	(	float *	buf,
		size_t	n,
		float	mean,
		float	stddev,
		uint32_t	seed )

pure virtualnoexcept

Fills n floats with samples from a normal distribution.

Parameters

buf	Buffer to fill.
n	Number of floats to fill.
mean	Distribution mean.
stddev	Distribution standard deviation.
seed	RNG seed.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ RandomizeUniform()

virtual void Deep::IComputeBackend::RandomizeUniform	(	float *	buf,
		size_t	n,
		float	min,
		float	max,
		uint32_t	seed )

pure virtualnoexcept

Fills n floats with samples from a uniform distribution over [min, max).

Parameters

buf	Buffer to fill.
n	Number of floats to fill.
min	Distribution lower bound (inclusive).
max	Distribution upper bound (exclusive).
seed	RNG seed.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ RepackForBatchedGemm()

virtual void Deep::IComputeBackend::RepackForBatchedGemm	(	float *	dst,
		const float *	src,
		size_t	batchSize,
		size_t	rows,
		size_t	cols )

pure virtualnoexcept

Repacks a [batchSize, rows, cols] tensor (batch-major) into [rows, batchSize, cols] (row-major, batch second) – i.e. dst[row][batch][col] = src[batch][row][col] for all row/batch/col, with the innermost cols dimension kept contiguous on both sides. Used by ConvPCLayer-family layers to reorganize per-batch im2col columns (and per-batch upstream-error columns) into the single flat layout a plain (non-batched) GEMM call needs, rather than one GEMM call per batch item. Same "port first, optimize" position as AddBiasBroadcast/SumRows: a future strided-batched-GEMM path could remove the need for this entirely, but this matches the existing, verified CPU repacking loops exactly for now.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ ReplayGraph()

virtual void Deep::IComputeBackend::ReplayGraph ( )

pure virtualnoexcept

Launches a graph previously captured and instantiated by BeginGraphCapture()/EndGraphCapture(). No-op if EndGraphCapture() never returned true.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Scale()

virtual void Deep::IComputeBackend::Scale	(	float *	buf,
		size_t	n,
		float	alpha )

pure virtualnoexcept

buf[i] *= alpha for all i in [0, n).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ SumRows()

virtual void Deep::IComputeBackend::SumRows	(	float *	dst,
		const float *	src,
		size_t	batchSize,
		size_t	width )

pure virtualnoexcept

dst[j] = sum over b in [0,batchSize) of src[b*width + j], for all j in [0,width). Replaces a batchSize-iteration loop of individual AxpyInto calls – the reduction-direction counterpart to AddBiasBroadcast, still unfixed until now. On GPU this is one cublasSgemv call against a cached all-ones vector, reinterpreting src's row-major [batchSize,width] layout as column-major [width,batchSize] with no data movement (verified numerically).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ TryFusedForwardPass()

virtual bool Deep::IComputeBackend::TryFusedForwardPass	(	ActivationType	actType,
		const float *	zF,
		const float *	W,
		const float *	bias,
		float *	mu,
		int	batchSize,
		int	size,
		int	nextSize )

pure virtualnoexcept

Attempts a fused forward pass (GEMM + bias + activation in one kernel, via CUTLASS on GPU) for RELU or LINEAR activation types only – see CUDABackend's implementation for why other activation types aren't attempted yet. Returns true if the fused path was used (mu is fully computed, including bias and activation); false if the caller should fall back to the existing MatMul+AddBiasBroadcast+ Activation sequence (always false on CPUBackend – CPU has no fused path, this is a GPU-only optimization).

Parameters

actType	Activation to fuse in; only RELU and LINEAR take the fused path, everything else returns false immediately.
zF	Activated input, shape [batchSize, size], row-major.
W	Weight matrix, shape [nextSize, size], row-major.
bias	Bias vector, shape [nextSize].
mu	Output, shape [batchSize, nextSize], row-major. Only written if this returns true.
batchSize	Number of rows in zF/mu.
size	Input width (columns of zF, columns of W).
nextSize	Output width (rows of W, columns of mu, length of bias).

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

◆ Zero()

virtual void Deep::IComputeBackend::Zero	(	float *	ptr,
		size_t	numFloats )

pure virtualnoexcept

Zero-fills numFloats floats starting at ptr.

Implemented in Deep::CPUBackend, and Deep::CUDABackend.

The documentation for this class was generated from the following file:

include/deepity/backend/IComputeBackend.h

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ ~IComputeBackend()

Member Function Documentation

◆ Activation()

◆ ActivationDerivative()

◆ ActivationDerivativeInto()

◆ ActivationInto()

◆ AdamStep()

◆ AdamWStep()

◆ AddBiasBroadcast()

◆ AddBiasPerChannel()

◆ Allocate()

◆ AxpyInto()

◆ BeginGraphCapture()

◆ Col2Im()

◆ ComputeError()

◆ ComputeErrorAndEnergy()

◆ ComputeSoftmaxCrossEntropyError()

◆ ComputeSoftmaxCrossEntropyErrorAndEnergy()

◆ Copy()

◆ CopyFromHost()

◆ CopyToHost()

◆ EndGraphCapture()

◆ Fill()

◆ Free()

◆ FusedStateUpdate()

◆ FusedStateUpdateMomentum()

◆ GetDeviceType()

◆ Im2Col()

◆ IncrementCounter()

◆ MatMul()

◆ MultiplyInto()

◆ PrepareForBatchSize()

◆ RandomizeNormal()

◆ RandomizeUniform()

◆ RepackForBatchedGemm()

◆ ReplayGraph()

◆ Scale()

◆ SumRows()

◆ TryFusedForwardPass()

◆ Zero()