returnn.frontend.gradient

Utilities which affect the gradient

returnn.frontend.gradient.set_requires_gradient(source: Tensor)

Parameters:

source

Returns:

nothing, modifies source in-place

returnn.frontend.gradient.gradient(y: Tensor, x: Tensor) → Tensor

Parameters:

• y – some scalar

• x – some tensor

Returns:

gradient of y w.r.t. x

returnn.frontend.gradient.stop_gradient(source: Tensor) → Tensor

wraps tf.stop_gradient or torch detach

returnn.frontend.gradient.scaled_gradient(source: Tensor, scale: float | Tensor) → Tensor

Parameters:

• source

• scale – if constant 0., will use stop_gradient(). Can be used as gradient reversal layer (with negative factor).

Returns:

source with scaled gradient

returnn.frontend.gradient.scaled_gradient_ext(source: Tensor, *, scale: float | Tensor, shift: float | Tensor | None = None, scale_shift_by_sum_over_axis: Dim | None = None) → Tensor

Just identity in the forward pass. Scales the gradient by some factor in backprop. Can be used as gradient reversal layer (with negative factor). For TF, uses returnn.tf.util.basic.scaled_gradient(), or tf.stop_gradient()

Parameters:

• source

• scale – if constant 0. and no shift, will use stop_gradient()

• shift

• scale_shift_by_sum_over_axis – if given, calculates the sum over this axis (absolute values) and multiplies the shift value by this sum.

Returns:

source with transformed gradient