Custom Layers

Eval Layer

class TFNetworkLayer.EvalLayer(eval, **kwargs)[source]

Evaluates some string. The CombineLayer provides this functionality, thus this is just a special case of it. Also see ActivationLayer.

Parameters:eval (str) – will eval this string. see _op_kind_eval()
layer_class = 'eval'[source]

Subnetwork Layer

class TFNetworkLayer.SubnetworkLayer(subnetwork, concat_sources=True, load_on_init=None, dropout=0, dropout_noise_shape=None, **kwargs)[source]

You can define a whole subnetwork as a single layer by this class.

The subnetwork will be specified by a dict[str,dict[str]], just like a normal network is specified in the config.

The "output" layer of the subnetwork will be the output of this subnetwork-layer.

With concat_sources=True (default),
the input to this layer will be represented as the "data:data" or simply "data" in the subnetwork,
otherwise with concat_sources=False,
the input to this layer will be represented as "data:input_layer_name" for each input, in the subnetwork.
  • subnetwork (dict[str,dict]) – subnetwork as dict (JSON content). must have an “output” layer-
  • concat_sources (bool) – if we concatenate all sources into one, like it is standard for most other layers
  • load_on_init (str|None) – if provided, for parameter initialization, we will load the given model file.
  • dropout (float) – will be applied if train_flag is set
  • dropout_noise_shape (tuple|list|dict|None) –
layer_class = 'subnetwork'[source]
recurrent = True[source]
classmethod get_out_data_from_opts(subnetwork, concat_sources=True, n_out=<class 'Util.NotSpecified'>, out_type=None, **kwargs)[source]
  • subnetwork (dict[str,dict[str]]) –
  • concat_sources (bool) –
  • n_out (int|None|NotSpecified) –
  • out_type (dict[str]|None) –
Return type:


Return type:tf.Tensor|None
classmethod get_losses(name, network, output, loss=None, reduce_func=None, layer=None, **kwargs)[source]
  • name (str) – layer name
  • network (TFNetwork.TFNetwork) –
  • loss (Loss|None) – argument just as for __init__
  • output (Data) – the output (template) for the layer
  • layer (LayerBase|None) –
  • reduce_func (((tf.Tensor)->tf.Tensor)|None) –
  • kwargs – other layer kwargs
Return type:


get_last_hidden_state(self, key)[source]
Parameters:key (int|str|None) – also the special key “*”
Return type:tf.Tensor|None
classmethod get_rec_initial_extra_outputs(batch_dim, rec_layer, subnetwork, **kwargs)[source]
  • batch_dim (tf.Tensor) – for this layer, might be with beam
  • rec_layer (TFNetworkRecLayer.RecLayer) –
  • subnetwork (dict[str,dict[str]]) –
Return type:


classmethod get_rec_initial_extra_outputs_shape_invariants(subnetwork, **kwargs)[source]
Parameters:subnetwork (dict[str,dict[str]]) –
Returns:optional shapes for the tensors by get_rec_initial_extra_outputs
Return type:dict[str,tf.TensorShape]