Custom Layers

Eval Layer

class, **kwargs)[source]

Evaluates some string. The CombineLayer provides this functionality, thus this is just a special case of it. Also see ActivationLayer, or CompareLayer.

The output type is defined as a broadcasted extension of all sources. You can overwrite it by (partially) specifying out_type. out_type can also be a generic Python function, returning a Data instance.

Parameters:eval (str) – will eval this string. see _op_kind_eval()
layer_class = 'eval'[source]

Subnetwork Layer

class, concat_sources=True, load_on_init=None, dropout=0, dropout_noise_shape=None, _parent_layer_cache=None, **kwargs)[source]

You can define a whole subnetwork as a single layer by this class.

The subnetwork will be specified by a dict[str,dict[str]], just like a normal network is specified in the config.

The "output" layer of the subnetwork will be the output of this subnetwork-layer.

With concat_sources=True (default),
the input to this layer will be represented as the "data:data" or simply "data" in the subnetwork,
otherwise with concat_sources=False,
the input to this layer will be represented as "data:input_layer_name" for each input, in the subnetwork.
  • subnetwork (dict[str,dict]) – subnetwork as dict (JSON content). must have an “output” layer-
  • concat_sources (bool) – if we concatenate all sources into one, like it is standard for most other layers
  • load_on_init (str|dict[str]|None) – if provided, for parameter initialization, we will load the given model file. see CustomCheckpointLoader.
  • dropout (float) – will be applied if train_flag is set
  • dropout_noise_shape (tuple|list|dict|None) –
  • _parent_layer_cache (dict[str,LayerBase]|None) –
layer_class = 'subnetwork'[source]
recurrent = True[source]
classmethod get_out_data_from_opts(subnetwork, concat_sources=True, n_out=<class 'returnn.util.basic.NotSpecified'>, out_type=None, **kwargs)[source]
  • subnetwork (dict[str,dict[str]]) –
  • concat_sources (bool) –
  • n_out (int|None|NotSpecified) –
  • out_type (dict[str]|None) –
Return type:


classmethod transform_config_dict(d, network, get_layer)[source]
Return type:tf.Tensor|None
classmethod get_losses(name, network, output, loss=None, reduce_func=None, layer=None, **kwargs)[source]
  • name (str) – layer name
  • network ( –
  • loss (Loss|None) – argument just as for __init__
  • output (Data) – the output (template) for the layer
  • layer (LayerBase|None) –
  • reduce_func (((tf.Tensor)->tf.Tensor)|None) –
  • kwargs – other layer kwargs
Return type:


Parameters:layer_name (str) – name of the sub_layer (right part of ‘/’ separated path)
Returns:the sub_layer addressed in layer_name or None if no sub_layer exists
Return type:LayerBase|None
Parameters:key (int|str|None) – also the special key “*”
Return type:tf.Tensor|None
classmethod get_rec_initial_extra_outputs(batch_dim, rec_layer, **kwargs)[source]
  • batch_dim (tf.Tensor) – for this layer, might be with beam
  • rec_layer (TFNetworkRecLayer.RecLayer) –
Return type:


classmethod get_rec_initial_extra_outputs_shape_invariants(**kwargs)[source]
Returns:optional shapes for the tensors by get_rec_initial_extra_outputs
Return type:dict[str,tf.TensorShape]