Generative Layers

Positional Encoding Layer

class returnn.tf.layers.rec.PositionalEncodingLayer(add_to_input=False, constant=-1, offset=None, **kwargs)[source]

Provides positional encoding in the form of (batch, time, n_out) or (time, batch, n_out) where n_out is the number of channels, if it is run outside a RecLayer, and (batch, n_out) or (n_out, batch) if run inside a RecLayer, where it will depend on the current time frame.

Assumes one source input with a time dimension if outside a RecLayer. With add_to_input, it will calculate x + input, and the output shape is the same as the input

The positional encoding is the same as in Tensor2Tensor. See TFUtil.get_positional_encoding().

Parameters:
  • add_to_input (bool) – will add the signal to the input
  • constant (int) – if positive, always output the corresponding positional encoding.
  • offset (None|LayerBase) – Specify the offset to be added to positions. Expect shape (batch, time) or (batch,).
layer_class = 'positional_encoding'[source]
recurrent = True[source]
classmethod transform_config_dict(d, network, get_layer)[source]
Parameters:
classmethod get_out_data_from_opts(name, network, add_to_input=False, sources=(), **kwargs)[source]
Parameters:
Return type:

Data

Range Layer

class returnn.tf.layers.basic.RangeLayer(limit, start=0, delta=1, dtype=None, sparse=False, **kwargs)[source]

Generic wrapper around tf.range. See also RangeInAxisLayer.

Parameters:
  • limit (int|float) –
  • start (int|float) –
  • delta (int|float) –
  • dtype (str|None) –
  • sparse (bool) –
layer_class = 'range'[source]
classmethod transform_config_dict(d, network, get_layer)[source]
Parameters:
classmethod get_out_data_from_opts(name, limit, start=0, delta=1, dtype=None, sparse=False, **kwargs)[source]
Parameters:
  • name (str) –
  • limit (int|float) –
  • start (int|float) –
  • delta (int|float) –
  • dtype (str|None) –
  • sparse (bool) –
Return type:

Data

Range in Axis Layer

class returnn.tf.layers.basic.RangeInAxisLayer(axis, dtype='int32', unbroadcast=False, keepdims=True, sparse=False, **kwargs)[source]

Assume that the input is e.g. (B,T,D), and you specify axis=”T”, you will get (B=1,T,D=1), where the specified axis is filled with tf.range. See also RangeLayer.

Parameters:
  • axis (str) –
  • dtype (str) –
  • unbroadcast (bool) –
  • keepdims (bool) –
  • sparse (bool) –
layer_class = 'range_in_axis'[source]
recurrent = True[source]
classmethod get_out_data_from_opts(name, sources, axis, dtype='int32', keepdims=True, sparse=False, **kwargs)[source]
Parameters:
  • name (str) –
  • sources (list[LayerBase]) –
  • axis (str) –
  • dtype (str) –
  • keepdims (bool) –
  • sparse (bool) –