returnn.tf.layers.variable

VariableLayer and related

class returnn.tf.layers.variable.VariableLayer(shape, dtype='float32', add_batch_axis=False, add_time_axis=False, trainable=True, saveable=True, non_critical_for_restore=False, init=None, init_by_layer=None, param_name=None, **kwargs)

Represents a variable. Can add batch/time dimension if wanted. Can be trainable. See defaults.

Parameters:

• shape (tuple[int|Dim]|list[int|Dim]) –

• dtype (str) –

• add_batch_axis (bool) –

• add_time_axis (bool) –

• trainable (bool) – whether it is updated by grad descent

• saveable (bool) – whether it is stored in the checkpoint

• non_critical_for_restore (bool) – if True, and it cannot be found in a checkpoint, it will not be an error

• init (str|float|int|None) – see returnn.tf.util.basic.get_initializer(). 0 by default. Alternatively, you can also use option init_by_layer.

• init_by_layer (LayerBase|None) –

• param_name (str|None) – self.name (layer name) by default

layer_class: Optional[str] = 'variable'

get_dep_layers()

Return type:

list[LayerBase]

classmethod transform_config_dict(d, network, get_layer)

Parameters:

• d (dict[str]) – will modify inplace

• network (returnn.tf.network.TFNetwork) –

• get_layer (((str) -> LayerBase)) – function to get or construct another layer

classmethod get_out_data_from_opts(name, network, shape, dtype='float32', add_batch_axis=False, add_time_axis=False, **kwargs)

Parameters:

• name (str) –

• network (returnn.tf.network.TFNetwork) –

• shape (tuple[int|Dim]|list[int|Dim]) –

• dtype (str) –

• add_batch_axis (bool) –

• add_time_axis (bool) –

Return type:

Tensor

kwargs: Optional[Dict[str]]

output_before_activation: Optional[OutputWithActivation]

output_loss: Optional[tf.Tensor]

rec_vars_outputs: Dict[str, tf.Tensor]

search_choices: Optional[SearchChoices]

params: Dict[str, tf.Variable]

saveable_param_replace: Dict[tf.Variable, Union['tensorflow.python.training.saver.BaseSaverBuilder.SaveableObject', None]]

stats: Dict[str, tf.Tensor]

class returnn.tf.layers.variable.VariableAssignLayer(var: LayerBase, value: LayerBase, control_dependencies: Sequence[LayerBase] | None = None, op: str = 'assign', **kwargs)

Assigns a new value to a variable.

Parameters:

• var –

• value –

• control_dependencies –

• op – “assign” or “add”

layer_class: Optional[str] = 'variable_assign'

get_dep_layers() → List[LayerBase]

deps

classmethod transform_config_dict(d, network, get_layer)

transform

classmethod get_out_data_from_opts(name: str, var: LayerBase, **kwargs)

out

kwargs: Optional[Dict[str]]

output_before_activation: Optional[OutputWithActivation]

output_loss: Optional[tf.Tensor]

rec_vars_outputs: Dict[str, tf.Tensor]

search_choices: Optional[SearchChoices]

params: Dict[str, tf.Variable]

saveable_param_replace: Dict[tf.Variable, Union['tensorflow.python.training.saver.BaseSaverBuilder.SaveableObject', None]]

stats: Dict[str, tf.Tensor]

class returnn.tf.layers.variable.VariableReadLayer(var: LayerBase, control_dependencies: Sequence[LayerBase] | None = None, **kwargs)

Read a variable (currently expected from VariableLayer). Supports control dependencies to exactly specify when it should be read.

Parameters:

• var – e.g. VariableLayer

• control_dependencies – to control what ops must run before the var is read (e.g. assign ops)

layer_class: Optional[str] = 'variable_read'

get_dep_layers() → List[LayerBase]

deps

classmethod transform_config_dict(d, network, get_layer)

transform

classmethod get_out_data_from_opts(name: str, var: LayerBase, **kwargs)

out

kwargs: Optional[Dict[str]]

output_before_activation: Optional[OutputWithActivation]

output_loss: Optional[tf.Tensor]

rec_vars_outputs: Dict[str, tf.Tensor]

search_choices: Optional[SearchChoices]

params: Dict[str, tf.Variable]

saveable_param_replace: Dict[tf.Variable, Union['tensorflow.python.training.saver.BaseSaverBuilder.SaveableObject', None]]

stats: Dict[str, tf.Tensor]