Custom Layers#

Eval Layer#

class returnn.tf.layers.basic.EvalLayer(eval, **kwargs)[source]#

Evaluates some string. The CombineLayer provides this functionality, thus this is just a special case of it. Also see ActivationLayer, or CompareLayer.

The output type is defined as a broadcasted extension of all sources. You can overwrite it by (partially) specifying out_type. out_type can also be a generic Python function, returning a Data instance.

Parameters:

eval (str) – will eval this string. see _op_kind_eval()

layer_class: Optional[str] = 'eval'[source]#
kwargs: Optional[Dict[str]][source]#
output_before_activation: Optional[OutputWithActivation][source]#
output_loss: Optional[tf.Tensor][source]#
rec_vars_outputs: Dict[str, tf.Tensor][source]#
search_choices: Optional[SearchChoices][source]#
params: Dict[str, tf.Variable][source]#
saveable_param_replace: Dict[tf.Variable, Union['tensorflow.python.training.saver.BaseSaverBuilder.SaveableObject', None]][source]#
stats: Dict[str, tf.Tensor][source]#

Subnetwork Layer#

class returnn.tf.layers.basic.SubnetworkLayer(subnetwork, _subnet, _output, concat_sources=True, load_on_init=None, dropout=0, dropout_noise_shape=None, _parent_layer_cache=None, _from=None, **kwargs)[source]#

You can define a whole subnetwork as a single layer by this class.

The subnetwork will be specified by a dict[str,dict[str]], just like a normal network is specified in the config.

The "output" layer of the subnetwork will be the output of this subnetwork-layer.

With concat_sources=True (default),

the input to this layer will be represented as the "data:data" or simply "data" in the subnetwork,

otherwise with concat_sources=False,

the input to this layer will be represented as "data:input_layer_name" and also "data:0" to "data:<n-1>" for n inputs, for each input, in the subnetwork. The first input will also be simply available as "data:data"/``”data”`.

Parameters:
  • subnetwork (dict[str,dict]) – subnetwork as dict (JSON content). must have an “output” layer-

  • concat_sources (bool) – if we concatenate all sources into one, like it is standard for most other layers

  • load_on_init (str|dict[str]|None) – if provided, for parameter initialization, we will load the given model file. see CustomCheckpointLoader.

  • dropout (float) – will be applied if train_flag is set

  • dropout_noise_shape (tuple|list|dict|None) –

  • _parent_layer_cache (dict[str,LayerBase]|None) –

  • _subnet (returnn.tf.network.Subnetwork) –

  • _output (LayerBase) –

layer_class: Optional[str] = 'subnetwork'[source]#
recurrent = True[source]#
update_params_from_subnet()[source]#

Update self.params.

update_rec_vars_outputs()[source]#

Update self.rec_vars_outputs.

update_load_on_init()[source]#

Handle load_on_init.

classmethod get_out_data_from_opts(n_out=<class 'returnn.util.basic.NotSpecified'>, out_type=None, **kwargs)[source]#
Parameters:
  • n_out (int|None|NotSpecified) –

  • out_type (dict[str]|None) –

Return type:

Data

classmethod transform_config_dict(d, network, get_layer)[source]#
Parameters:
classmethod get_sub_layer_out_data_from_opts(layer_name, parent_layer_kwargs)[source]#
Parameters:
  • layer_name (str) – name of the sub_layer (right part of ‘/’ separated path)

  • parent_layer_kwargs (dict[str]) – kwargs for the parent layer (as kwargs in cls.get_out_data_from_opts())

Returns:

Data template, class type of sub-layer, layer opts (transformed)

Return type:

(Data, type, dict[str])|None

classmethod cls_get_sub_network(name, network, layer_desc)[source]#
Parameters:
Return type:

returnn.tf.network.Subnetwork|None

get_sub_layer(layer_name)[source]#
Parameters:

layer_name (str) – name of the sub_layer (right part of ‘/’ separated path)

Returns:

the sub_layer addressed in layer_name or None if no sub_layer exists

Return type:

LayerBase|None

classmethod get_available_sub_layer_names(parent_layer_kwargs)[source]#
Parameters:

parent_layer_kwargs (dict[str]) –

Return type:

list[str]

get_sub_networks()[source]#
Return type:

list[returnn.tf.network.TFNetwork]

get_sub_layers()[source]#
Return type:

list[LayerBase]

get_dep_layers()[source]#
Returns:

list of layers this layer depends on. normally this is just self.sources but e.g. the attention layer in addition has a base, etc.

Return type:

list[LayerBase]

get_last_hidden_state(key)[source]#
Parameters:

key (int|str|None) – also the special key “*”

Return type:

tf.Tensor|None

classmethod get_rec_initial_extra_outputs(batch_dim, rec_layer, encapsulate=False, **kwargs)[source]#
Parameters:
Return type:

dict[str,tf.Tensor]

classmethod get_rec_initial_extra_outputs_shape_invariants(rec_layer, encapsulate=False, **kwargs)[source]#
Parameters:
Returns:

optional shapes for the tensors by get_rec_initial_extra_outputs

Return type:

dict[str,tf.TensorShape]

kwargs: Optional[Dict[str]][source]#
output_before_activation: Optional[OutputWithActivation][source]#
output_loss: Optional[tf.Tensor][source]#
rec_vars_outputs: Dict[str, tf.Tensor][source]#
search_choices: Optional[SearchChoices][source]#
params: Dict[str, tf.Variable][source]#
saveable_param_replace: Dict[tf.Variable, Union['tensorflow.python.training.saver.BaseSaverBuilder.SaveableObject', None]][source]#
stats: Dict[str, tf.Tensor][source]#