Relu.h File Reference

ReLU and its derivative (plain and two-buffer variants), with AVX512/AVX2/SSE paths and a scalar tail. More...

#include <cassert>
#include <cstddef>
#include <immintrin.h>
#include <omp.h>
#include <deepity/utils/activations/ActivationMacros.h>

Go to the source code of this file.

Namespaces
namespace	Deep
	This file exists to provide both Tensor.h and IComputeBackend.h with device types without introducing circular dependencies.

Macros
#define	MAX(a, b)
	Guarded so this doesn't clash with a system MAX macro (notably Windows headers) if one is already defined.

Functions
static void	Deep::relu (float *RESTRICT x, const size_t n) noexcept
	RELU(x) = MAX(0, x) for all x.
static void	Deep::dRelu (float *RESTRICT x, const size_t n, bool activated=false) noexcept
	In-place ReLU derivative: 1 where x > 0, else 0.
static void	Deep::dReluInto (float *RESTRICT dst, const float *RESTRICT src, const size_t n) noexcept
	Two-buffer variant of dRelu: reads src, writes the derivative directly into dst.

Detailed Description

ReLU and its derivative (plain and two-buffer variants), with AVX512/AVX2/SSE paths and a scalar tail.

Macro Definition Documentation

MAX

#define MAX	(		a,
			b )

Value:

(((a) > (b)) ? (a) : (b))

Guarded so this doesn't clash with a system MAX macro (notably Windows headers) if one is already defined.