Deep::CUDABackend Class Reference

CUDA implementation of IComputeBackend: device kernels for elementwise ops, cuBLAS for GEMM, CUTLASS for the fused forward path, CUDA graphs for capture/replay. See src/backend/cuda/ for the implementation, split by concern to mirror this interface's own section grouping. More...

#include <CUDABackend.h>

Inheritance diagram for Deep::CUDABackend:

Public Member Functions
	CUDABackend ()
	~CUDABackend () override
void	BeginGraphCapture () noexcept override
	Starts recording subsequent backend calls into a replayable graph, instead of executing them immediately (CUDA graph capture; a no-op record on CPUBackend). Pair with EndGraphCapture().
bool	EndGraphCapture () noexcept override
	Ends capture and instantiates the captured graph.
void	ReplayGraph () noexcept override
	Launches a graph previously captured and instantiated by BeginGraphCapture()/EndGraphCapture(). No-op if EndGraphCapture() never returned true.
float *	Allocate (size_t numFloats) override
	Allocates numFloats floats on this backend's device.
void	Free (float *ptr) noexcept override
	Frees a buffer previously returned by Allocate(). No-op on nullptr.
void	Zero (float *ptr, size_t numFloats) noexcept override
	Zero-fills numFloats floats starting at ptr.
void	Copy (float *dst, const float *src, size_t numFloats) noexcept override
	Copies numFloats floats from src to dst, both on this backend's own device.
void	CopyFromHost (float *deviceDst, const float *hostSrc, size_t numFloats) noexcept override
	Copies numFloats floats from a host buffer into a device buffer (a plain memcpy on CPUBackend).
void	CopyToHost (float *hostDst, const float *deviceSrc, size_t numFloats) noexcept override
	Copies numFloats floats from a device buffer into a host buffer (a plain memcpy on CPUBackend).
void	RandomizeNormal (float *buf, size_t n, float mean, float stddev, uint32_t seed) noexcept override
	Fills n floats with samples from a normal distribution.
void	RandomizeUniform (float *buf, size_t n, float min, float max, uint32_t seed) noexcept override
	Fills n floats with samples from a uniform distribution over [min, max).
void	SumRows (float *dst, const float *src, size_t batchSize, size_t width) noexcept override
	dst[j] = sum over b in [0,batchSize) of src[b*width + j], for all j in [0,width). Replaces a batchSize-iteration loop of individual AxpyInto calls – the reduction-direction counterpart to AddBiasBroadcast, still unfixed until now. On GPU this is one cublasSgemv call against a cached all-ones vector, reinterpreting src's row-major [batchSize,width] layout as column-major [width,batchSize] with no data movement (verified numerically).
void	PrepareForBatchSize (size_t batchSize) noexcept override
	Must be called once, before Compile()'s first BeginGraphCapture(), for any backend that needs to prepare batch-size-dependent state (e.g. CUDABackend's cached all-ones vector for SumRows' GEMV). No-op on CPUBackend.
void	MatMul (bool transA, bool transB, int M, int N, int K, float alpha, const float *A, int lda, const float *B, int ldb, float beta, float *C, int ldc) noexcept override
	C = alpha * op(A) * op(B) + beta * C, the standard BLAS SGEMM convention (op(X) = X^T if the matching transX is true).
void	Scale (float *buf, size_t n, float alpha) noexcept override
	buf[i] *= alpha for all i in [0, n).
void	AxpyInto (float *y, const float *x, size_t n, float alpha) noexcept override
	y[i] += alpha * x[i] for all i in [0, n) – the standard AXPY.
void	AddBiasBroadcast (float *buf, const float *bias, size_t batchSize, size_t width) noexcept override
	buf[row*width + col] += bias[col] for every row in [0,batchSize) and col in [0,width) – broadcasts a per-column bias across every row (the dense-layer convention).
void	FusedStateUpdateMomentum (float *z, float *v, const float *feedback, const float *deriv, const float *e, size_t n, float ir, float beta) noexcept override
	Same as FusedStateUpdate, but with momentum: the update direction (feedback*deriv - e) is EMA-smoothed into v (same shape as z, persistent across settling steps within one TrainStep(), reset to zero at the start of each call) before being applied to z. beta is the EMA decay (0.9 is a common default – higher = more smoothing/inertia). v = beta*v + (1-beta)*((feedback*deriv) - e) z += ir*v.
bool	TryFusedForwardPass (ActivationType actType, const float *zF, const float *W, const float *bias, float *mu, int batchSize, int size, int nextSize) noexcept override
	Attempts a fused forward pass (GEMM + bias + activation in one kernel, via CUTLASS on GPU) for RELU or LINEAR activation types only – see CUDABackend's implementation for why other activation types aren't attempted yet. Returns true if the fused path was used (mu is fully computed, including bias and activation); false if the caller should fall back to the existing MatMul+AddBiasBroadcast+ Activation sequence (always false on CPUBackend – CPU has no fused path, this is a GPU-only optimization).
void	Activation (ActivationType type, float *buf, size_t n) noexcept override
	Applies type to buf in place.
void	ActivationInto (ActivationType type, float *dst, const float *src, size_t n) noexcept override
	Two-buffer variant of Activation(): reads src, writes into dst.
void	ActivationDerivative (ActivationType type, float *buf, size_t n, bool activated) noexcept override
	Applies type's derivative to buf in place.
void	ActivationDerivativeInto (ActivationType type, float *dst, const float *src, size_t n) noexcept override
	Two-buffer variant of ActivationDerivative(): reads src (the pre-activation value), writes the derivative into dst.
void	FusedStateUpdate (float *z, const float *feedback, const float *deriv, const float *e, size_t n, float ir) noexcept override
	One settling step's state update, fused into a single call: z += ir * ((feedback * deriv) - e) Non-momentum counterpart to FusedStateUpdateMomentum() above.
float	ComputeErrorAndEnergy (float *e, const float *z, const float *mu, size_t n) noexcept override
	Computes e = z - mu, then returns 0.5 * sum(e^2) – the Gaussian error/energy used by every layer except a cross-entropy terminal.
void	ComputeError (float *e, const float *z, const float *mu, size_t n) noexcept override
	Same as ComputeErrorAndEnergy(), without computing (or returning) the energy – for callers that only need e.
float	ComputeSoftmaxCrossEntropyErrorAndEnergy (float *e, const float *z, const float *mu, size_t batchSize, size_t nextSize, float *rowEnergies) noexcept override
	Softmax cross-entropy variant of ComputeErrorAndEnergy, for the terminal layer only. Unlike the Gaussian version, this needs batchSize/nextSize separately (not just a flat n) since softmax is a row-wise, not element-wise, operation. probs = softmax(mu), row-wise e = z - probs (z is the clamped target; matches the existing e := -dF/dmu convention exactly, since d(CE)/d(logits) = probs - y) energy = -sum(z * log(probs + eps)) (eps=1e-8, avoids log(0)) mu holds the RAW LOGITS (unchanged from the Gaussian case, still a*W@phi(z)+b) – softmax is applied here, not baked into mu itself. rowEnergies is caller-provided scratch, >= batchSize floats – NOT allocated internally (this runs inside CalculateState(), which runs inside TrainStep()'s CUDA-graph-captured region; dynamic allocation there is unsafe under capture, same reasoning as the CUTLASS workspace). CPUBackend's own override ignores this parameter entirely – kept in the shared interface so callers pass the same arguments to either backend.
void	ComputeSoftmaxCrossEntropyError (float *e, const float *z, const float *mu, size_t batchSize, size_t nextSize) noexcept override
	Same as ComputeSoftmaxCrossEntropyErrorAndEnergy(), without computing (or requiring rowEnergies for) the energy – for callers that only need e.
void	IncrementCounter (int *ptr) noexcept override
	Atomically increments *counter by 1. Device-resident so it can be read/updated by the same kernel/graph across settling steps (e.g. Adam's step count t) without a host round-trip.
void	AdamStep (float *param, const float *grad, float *m, float *v, size_t n, const int *t, const float *lr, float beta1=0.9f, float beta2=0.999f, float eps=1e-8f) noexcept override
	Adam update for n parameters in place.
void	AdamWStep (float *param, const float *grad, float *m, float *v, size_t n, const int *t, const float *lr, float weightDecay, float beta1=0.9f, float beta2=0.999f, float eps=1e-8f) noexcept override
	Same as AdamStep(), plus decoupled weight decay (AdamW): param -= lr * weightDecay * param, applied before the Adam step itself.
DeviceType	GetDeviceType () const noexcept override
	Which device this backend instance runs on.
void	MultiplyInto (float *dst, const float *a, const float *b, size_t n) noexcept override
	dst[i] = a[i] * b[i] for all i in [0, n) – elementwise product.
void	Fill (float *buf, size_t n, float value) noexcept override
	buf[i] = value for all i in [0, n).
void	Im2Col (const float *input, int channels, int height, int width, int kernelH, int kernelW, int strideH, int strideW, int padH, int padW, float *columns) noexcept override
	Rearranges a single (channels, height, width) input image into a (channels*kH*kW, outH*outW) column matrix – the standard im2col transform. NOT batch-aware: call once per batch item, with input and columns offset to that item's slice, matching Deep::Im2Col's own documented contract exactly (CPUBackend forwards to it directly). Positions outside the input (due to padding) are written as zero. columns is fully overwritten, not accumulated into.
void	Col2Im (const float *columns, int channels, int height, int width, int kernelH, int kernelW, int strideH, int strideW, int padH, int padW, float *outputImage) noexcept override
	The adjoint of Im2Col(): scatters a (channels*kH*kW, outH*outW) column-gradient buffer back into a (channels, height, width) image. NOT batch-aware, same per-item-offset contract as Im2Col(). ACCUMULATES into outputImage (does not zero it first) – caller must zero the destination if a fresh result is wanted, matching Deep::Col2Im's own contract exactly.
void	RepackForBatchedGemm (float *dst, const float *src, size_t batchSize, size_t rows, size_t cols) noexcept override
	Repacks a [batchSize, rows, cols] tensor (batch-major) into [rows, batchSize, cols] (row-major, batch second) – i.e. dst[row][batch][col] = src[batch][row][col] for all row/batch/col, with the innermost cols dimension kept contiguous on both sides. Used by ConvPCLayer-family layers to reorganize per-batch im2col columns (and per-batch upstream-error columns) into the single flat layout a plain (non-batched) GEMM call needs, rather than one GEMM call per batch item. Same "port first, optimize" position as AddBiasBroadcast/SumRows: a future strided-batched-GEMM path could remove the need for this entirely, but this matches the existing, verified CPU repacking loops exactly for now.
void	AddBiasPerChannel (float *buf, const float *bias, size_t channels, size_t spatialSize) noexcept override
	Convolutional bias-add: buf[c*spatialSize + s] += bias[c] for all c in [0,channels), s in [0,spatialSize). Per-CHANNEL broadcast across spatial positions – the transpose relationship to AddBiasBroadcast (which broadcasts a per-COLUMN bias across ROWS, the dense-layer convention). NOT batch-aware: matches Im2Col/ Col2Im's contract exactly – call once per batch item, with buf offset to that item's slice.
Public Member Functions inherited from Deep::IComputeBackend
virtual	~IComputeBackend ()=default

Detailed Description

CUDA implementation of IComputeBackend: device kernels for elementwise ops, cuBLAS for GEMM, CUTLASS for the fused forward path, CUDA graphs for capture/replay. See src/backend/cuda/ for the implementation, split by concern to mirror this interface's own section grouping.

Constructor & Destructor Documentation

CUDABackend()

Deep::CUDABackend::CUDABackend

(

)

~CUDABackend()

Deep::CUDABackend::~CUDABackend ( )

override

Member Function Documentation

Activation()

void Deep::CUDABackend::Activation ( ActivationType type, float * buf, size_t n )

overridevirtualnoexcept

Applies type to buf in place.

Implements Deep::IComputeBackend.

ActivationDerivative()

void Deep::CUDABackend::ActivationDerivative ( ActivationType type, float * buf, size_t n, bool activated )

overridevirtualnoexcept

Applies type's derivative to buf in place.

Parameters

type	Which activation's derivative to apply.
buf	Array to derive in place.
n	Length of buf.
activated	If true, buf already holds the activated value; if false, buf holds the pre-activation value.

Implements Deep::IComputeBackend.

ActivationDerivativeInto()

void Deep::CUDABackend::ActivationDerivativeInto ( ActivationType type, float * dst, const float * src, size_t n )

overridevirtualnoexcept

Two-buffer variant of ActivationDerivative(): reads src (the pre-activation value), writes the derivative into dst.

Implements Deep::IComputeBackend.

ActivationInto()

void Deep::CUDABackend::ActivationInto ( ActivationType type, float * dst, const float * src, size_t n )

overridevirtualnoexcept

Two-buffer variant of Activation(): reads src, writes into dst.

Implements Deep::IComputeBackend.

AdamStep()

void Deep::CUDABackend::AdamStep ( float * param, const float * grad, float * m, float * v, size_t n, const int * t, const float * lr, float beta1 = 0.9f, float beta2 = 0.999f, float eps = 1e-8f )

overridevirtualnoexcept

Adam update for n parameters in place.

Parameters

param	Parameters to update.
grad	Gradient w.r.t. param.
m	Adam's first-moment (momentum) buffer, persistent across calls.
v	Adam's second-moment (variance) buffer, persistent across calls.
n	Number of parameters.
t	Device-resident step count (read, not written, here – see IncrementCounter()).
lr	Device-resident learning rate.
beta1,beta2	Momentum/variance EMA decay rates.
eps	Denominator stabilizer.

Implements Deep::IComputeBackend.

AdamWStep()

void Deep::CUDABackend::AdamWStep ( float * param, const float * grad, float * m, float * v, size_t n, const int * t, const float * lr, float weightDecay, float beta1 = 0.9f, float beta2 = 0.999f, float eps = 1e-8f )

overridevirtualnoexcept

Same as AdamStep(), plus decoupled weight decay (AdamW): param -= lr * weightDecay * param, applied before the Adam step itself.

Implements Deep::IComputeBackend.

AddBiasBroadcast()

void Deep::CUDABackend::AddBiasBroadcast ( float * buf, const float * bias, size_t batchSize, size_t width )

overridevirtualnoexcept

buf[row*width + col] += bias[col] for every row in [0,batchSize) and col in [0,width) – broadcasts a per-column bias across every row (the dense-layer convention).

Implements Deep::IComputeBackend.

AddBiasPerChannel()

void Deep::CUDABackend::AddBiasPerChannel ( float * buf, const float * bias, size_t channels, size_t spatialSize )

overridevirtualnoexcept

Convolutional bias-add: buf[c*spatialSize + s] += bias[c] for all c in [0,channels), s in [0,spatialSize). Per-CHANNEL broadcast across spatial positions – the transpose relationship to AddBiasBroadcast (which broadcasts a per-COLUMN bias across ROWS, the dense-layer convention). NOT batch-aware: matches Im2Col/ Col2Im's contract exactly – call once per batch item, with buf offset to that item's slice.

Implements Deep::IComputeBackend.

Allocate()

float * Deep::CUDABackend::Allocate ( size_t numFloats )

overridevirtual

Allocates numFloats floats on this backend's device.

Returns

nullptr on allocation failure.

Implements Deep::IComputeBackend.

AxpyInto()

void Deep::CUDABackend::AxpyInto ( float * y, const float * x, size_t n, float alpha )

overridevirtualnoexcept

y[i] += alpha * x[i] for all i in [0, n) – the standard AXPY.

Implements Deep::IComputeBackend.

BeginGraphCapture()

void Deep::CUDABackend::BeginGraphCapture ( )

overridevirtualnoexcept

Starts recording subsequent backend calls into a replayable graph, instead of executing them immediately (CUDA graph capture; a no-op record on CPUBackend). Pair with EndGraphCapture().

Implements Deep::IComputeBackend.

Col2Im()

void Deep::CUDABackend::Col2Im ( const float * columns, int channels, int height, int width, int kernelH, int kernelW, int strideH, int strideW, int padH, int padW, float * outputImage )

overridevirtualnoexcept

The adjoint of Im2Col(): scatters a (channels*kH*kW, outH*outW) column-gradient buffer back into a (channels, height, width) image. NOT batch-aware, same per-item-offset contract as Im2Col(). ACCUMULATES into outputImage (does not zero it first) – caller must zero the destination if a fresh result is wanted, matching Deep::Col2Im's own contract exactly.

Implements Deep::IComputeBackend.

ComputeError()

void Deep::CUDABackend::ComputeError ( float * e, const float * z, const float * mu, size_t n )

overridevirtualnoexcept

Same as ComputeErrorAndEnergy(), without computing (or returning) the energy – for callers that only need e.

Implements Deep::IComputeBackend.

ComputeErrorAndEnergy()

float Deep::CUDABackend::ComputeErrorAndEnergy ( float * e, const float * z, const float * mu, size_t n )

overridevirtualnoexcept

Computes e = z - mu, then returns 0.5 * sum(e^2) – the Gaussian error/energy used by every layer except a cross-entropy terminal.

Implements Deep::IComputeBackend.

ComputeSoftmaxCrossEntropyError()

void Deep::CUDABackend::ComputeSoftmaxCrossEntropyError ( float * e, const float * z, const float * mu, size_t batchSize, size_t nextSize )

overridevirtualnoexcept

Same as ComputeSoftmaxCrossEntropyErrorAndEnergy(), without computing (or requiring rowEnergies for) the energy – for callers that only need e.

Implements Deep::IComputeBackend.

ComputeSoftmaxCrossEntropyErrorAndEnergy()

float Deep::CUDABackend::ComputeSoftmaxCrossEntropyErrorAndEnergy ( float * e, const float * z, const float * mu, size_t batchSize, size_t nextSize, float * rowEnergies )

overridevirtualnoexcept

Softmax cross-entropy variant of ComputeErrorAndEnergy, for the terminal layer only. Unlike the Gaussian version, this needs batchSize/nextSize separately (not just a flat n) since softmax is a row-wise, not element-wise, operation. probs = softmax(mu), row-wise e = z - probs (z is the clamped target; matches the existing e := -dF/dmu convention exactly, since d(CE)/d(logits) = probs - y) energy = -sum(z * log(probs + eps)) (eps=1e-8, avoids log(0)) mu holds the RAW LOGITS (unchanged from the Gaussian case, still a*W@phi(z)+b) – softmax is applied here, not baked into mu itself. rowEnergies is caller-provided scratch, >= batchSize floats – NOT allocated internally (this runs inside CalculateState(), which runs inside TrainStep()'s CUDA-graph-captured region; dynamic allocation there is unsafe under capture, same reasoning as the CUTLASS workspace). CPUBackend's own override ignores this parameter entirely – kept in the shared interface so callers pass the same arguments to either backend.

Implements Deep::IComputeBackend.

Copy()

void Deep::CUDABackend::Copy ( float * dst, const float * src, size_t numFloats )

overridevirtualnoexcept

Copies numFloats floats from src to dst, both on this backend's own device.

Implements Deep::IComputeBackend.

CopyFromHost()

void Deep::CUDABackend::CopyFromHost ( float * deviceDst, const float * hostSrc, size_t numFloats )

overridevirtualnoexcept

Copies numFloats floats from a host buffer into a device buffer (a plain memcpy on CPUBackend).

Implements Deep::IComputeBackend.

CopyToHost()

void Deep::CUDABackend::CopyToHost ( float * hostDst, const float * deviceSrc, size_t numFloats )

overridevirtualnoexcept

Copies numFloats floats from a device buffer into a host buffer (a plain memcpy on CPUBackend).

Implements Deep::IComputeBackend.

EndGraphCapture()

bool Deep::CUDABackend::EndGraphCapture ( )

overridevirtualnoexcept

Ends capture and instantiates the captured graph.

Returns

true if capture and instantiation both succeeded and ReplayGraph() is now safe to call; false otherwise. Callers MUST check this – silently assuming success here was the cause of a real bug: a failed capture left ReplayGraph() permanently doing nothing on every subsequent call, since the caller had no way to know capture never actually happened.

Implements Deep::IComputeBackend.

Fill()

void Deep::CUDABackend::Fill ( float * buf, size_t n, float value )

overridevirtualnoexcept

buf[i] = value for all i in [0, n).

Implements Deep::IComputeBackend.

Free()

void Deep::CUDABackend::Free ( float * ptr )

overridevirtualnoexcept

Frees a buffer previously returned by Allocate(). No-op on nullptr.

Implements Deep::IComputeBackend.

FusedStateUpdate()

void Deep::CUDABackend::FusedStateUpdate ( float * z, const float * feedback, const float * deriv, const float * e, size_t n, float ir )

overridevirtualnoexcept

One settling step's state update, fused into a single call: z += ir * ((feedback * deriv) - e) Non-momentum counterpart to FusedStateUpdateMomentum() above.

Implements Deep::IComputeBackend.

FusedStateUpdateMomentum()

void Deep::CUDABackend::FusedStateUpdateMomentum ( float * z, float * v, const float * feedback, const float * deriv, const float * e, size_t n, float ir, float beta )

overridevirtualnoexcept

Same as FusedStateUpdate, but with momentum: the update direction (feedback*deriv - e) is EMA-smoothed into v (same shape as z, persistent across settling steps within one TrainStep(), reset to zero at the start of each call) before being applied to z. beta is the EMA decay (0.9 is a common default – higher = more smoothing/inertia). v = beta*v + (1-beta)*((feedback*deriv) - e) z += ir*v.

Implements Deep::IComputeBackend.

GetDeviceType()

DeviceType Deep::CUDABackend::GetDeviceType ( ) const

inlineoverridevirtualnoexcept

Which device this backend instance runs on.

Implements Deep::IComputeBackend.

Im2Col()

void Deep::CUDABackend::Im2Col ( const float * input, int channels, int height, int width, int kernelH, int kernelW, int strideH, int strideW, int padH, int padW, float * columns )

overridevirtualnoexcept

Rearranges a single (channels, height, width) input image into a (channels*kH*kW, outH*outW) column matrix – the standard im2col transform. NOT batch-aware: call once per batch item, with input and columns offset to that item's slice, matching Deep::Im2Col's own documented contract exactly (CPUBackend forwards to it directly). Positions outside the input (due to padding) are written as zero. columns is fully overwritten, not accumulated into.

Implements Deep::IComputeBackend.

IncrementCounter()

void Deep::CUDABackend::IncrementCounter ( int * ptr )

overridevirtualnoexcept

Atomically increments *counter by 1. Device-resident so it can be read/updated by the same kernel/graph across settling steps (e.g. Adam's step count t) without a host round-trip.

Implements Deep::IComputeBackend.

MatMul()

void Deep::CUDABackend::MatMul ( bool transA, bool transB, int M, int N, int K, float alpha, const float * A, int lda, const float * B, int ldb, float beta, float * C, int ldc )

overridevirtualnoexcept

C = alpha * op(A) * op(B) + beta * C, the standard BLAS SGEMM convention (op(X) = X^T if the matching transX is true).

Parameters

transA	Whether to transpose A.
transB	Whether to transpose B.
M	Rows of op(A) and C.
N	Columns of op(B) and C.
K	Columns of op(A), rows of op(B).
alpha	Scalar multiplied into op(A)*op(B).
A	Left operand.
lda,ldb,ldc	Leading dimensions of A, B, C respectively.
B	Right operand.
beta	Scalar multiplied into the existing C before accumulating.
C	Output; also read if beta != 0.

Implements Deep::IComputeBackend.

MultiplyInto()

void Deep::CUDABackend::MultiplyInto ( float * dst, const float * a, const float * b, size_t n )

overridevirtualnoexcept

dst[i] = a[i] * b[i] for all i in [0, n) – elementwise product.

Implements Deep::IComputeBackend.

PrepareForBatchSize()

void Deep::CUDABackend::PrepareForBatchSize ( size_t batchSize )

overridevirtualnoexcept

Must be called once, before Compile()'s first BeginGraphCapture(), for any backend that needs to prepare batch-size-dependent state (e.g. CUDABackend's cached all-ones vector for SumRows' GEMV). No-op on CPUBackend.

Implements Deep::IComputeBackend.

RandomizeNormal()

void Deep::CUDABackend::RandomizeNormal ( float * buf, size_t n, float mean, float stddev, uint32_t seed )

overridevirtualnoexcept

Fills n floats with samples from a normal distribution.

Parameters

buf	Buffer to fill.
n	Number of floats to fill.
mean	Distribution mean.
stddev	Distribution standard deviation.
seed	RNG seed.

Implements Deep::IComputeBackend.

RandomizeUniform()

void Deep::CUDABackend::RandomizeUniform ( float * buf, size_t n, float min, float max, uint32_t seed )

overridevirtualnoexcept

Fills n floats with samples from a uniform distribution over [min, max).

Parameters

buf	Buffer to fill.
n	Number of floats to fill.
min	Distribution lower bound (inclusive).
max	Distribution upper bound (exclusive).
seed	RNG seed.

Implements Deep::IComputeBackend.

RepackForBatchedGemm()

void Deep::CUDABackend::RepackForBatchedGemm ( float * dst, const float * src, size_t batchSize, size_t rows, size_t cols )

overridevirtualnoexcept

Repacks a [batchSize, rows, cols] tensor (batch-major) into [rows, batchSize, cols] (row-major, batch second) – i.e. dst[row][batch][col] = src[batch][row][col] for all row/batch/col, with the innermost cols dimension kept contiguous on both sides. Used by ConvPCLayer-family layers to reorganize per-batch im2col columns (and per-batch upstream-error columns) into the single flat layout a plain (non-batched) GEMM call needs, rather than one GEMM call per batch item. Same "port first, optimize" position as AddBiasBroadcast/SumRows: a future strided-batched-GEMM path could remove the need for this entirely, but this matches the existing, verified CPU repacking loops exactly for now.

Implements Deep::IComputeBackend.

ReplayGraph()

void Deep::CUDABackend::ReplayGraph ( )

overridevirtualnoexcept

Launches a graph previously captured and instantiated by BeginGraphCapture()/EndGraphCapture(). No-op if EndGraphCapture() never returned true.

Implements Deep::IComputeBackend.

Scale()

void Deep::CUDABackend::Scale ( float * buf, size_t n, float alpha )

overridevirtualnoexcept

buf[i] *= alpha for all i in [0, n).

Implements Deep::IComputeBackend.

SumRows()

void Deep::CUDABackend::SumRows ( float * dst, const float * src, size_t batchSize, size_t width )

overridevirtualnoexcept

dst[j] = sum over b in [0,batchSize) of src[b*width + j], for all j in [0,width). Replaces a batchSize-iteration loop of individual AxpyInto calls – the reduction-direction counterpart to AddBiasBroadcast, still unfixed until now. On GPU this is one cublasSgemv call against a cached all-ones vector, reinterpreting src's row-major [batchSize,width] layout as column-major [width,batchSize] with no data movement (verified numerically).

Implements Deep::IComputeBackend.

TryFusedForwardPass()

bool Deep::CUDABackend::TryFusedForwardPass ( ActivationType actType, const float * zF, const float * W, const float * bias, float * mu, int batchSize, int size, int nextSize )

overridevirtualnoexcept

Attempts a fused forward pass (GEMM + bias + activation in one kernel, via CUTLASS on GPU) for RELU or LINEAR activation types only – see CUDABackend's implementation for why other activation types aren't attempted yet. Returns true if the fused path was used (mu is fully computed, including bias and activation); false if the caller should fall back to the existing MatMul+AddBiasBroadcast+ Activation sequence (always false on CPUBackend – CPU has no fused path, this is a GPU-only optimization).

Parameters

actType	Activation to fuse in; only RELU and LINEAR take the fused path, everything else returns false immediately.
zF	Activated input, shape [batchSize, size], row-major.
W	Weight matrix, shape [nextSize, size], row-major.
bias	Bias vector, shape [nextSize].
mu	Output, shape [batchSize, nextSize], row-major. Only written if this returns true.
batchSize	Number of rows in zF/mu.
size	Input width (columns of zF, columns of W).
nextSize	Output width (rows of W, columns of mu, length of bias).

Implements Deep::IComputeBackend.

Zero()

void Deep::CUDABackend::Zero ( float * ptr, size_t numFloats )

overridevirtualnoexcept

Zero-fills numFloats floats starting at ptr.

Implements Deep::IComputeBackend.

---

The documentation for this class was generated from the following file:

• include/deepity/backend/CUDABackend.h