This module contains pretrain related code. This is independent from the backend (TF or Theano, etc).

class Pretrain.WrapEpochValue(func)[source]

Use this wrapper if you want to define some value in your network which depends on the pretrain epoch. This is going to be part in your network description dict.

Parameters:((epoch – int) -> object) func: function which should accept one kwd-arg ‘epoch’
get_value(self, epoch)[source]
Parameters:epoch (int) –
Returns:anything, whatever self.func returns
Return type:object

See also Pretrain._resolve_wrapped_values(). Recursively goes through dicts, tuples and lists. This is a simple check to see if this is needed, i.e. if there are any WrapEpochValue used.

Parameters:net_json (dict[str]) – network dict
Returns:whether there is some WrapEpochValue in it
Return type:bool
class Pretrain.Pretrain(original_network_json, network_init_args=None, copy_param_mode=None, copy_output_layer=None, greedy=None, repetitions=None, construction_algo='from_output', output_layers=('output', ), input_layers=('data', ))[source]

Start with 1 hidden layers up to N hidden layers -> N pretrain steps -> N epochs (with repetitions == 1). The first hidden layer is the input layer. This works for generic network constructions. See _construct_epoch().

  • network_init_args (dict[str]|None) – additional args we use for LayerNetwork.from_json(). must have n_in, n_out. (for Theano only, thus optional now)
  • copy_param_mode (str) –
  • copy_output_layer (bool|str) – whether to copy the output layer params from last epoch or reinit
  • greedy (bool) – if True, only train output+last layer, otherwise train all
  • | int | list[int] | dict repetitions (None) – how often to repeat certain pretrain steps. default is one epoch. It can also be a dict, with keys like ‘default’ and ‘final’. See code below.
  • construction_algo (str|callable) – e.g. “from_output”
  • output_layers (list[str]|tuple[str]) – used for construction
  • input_layers (list[str]|tuple[str]) – used for construction
Return type:int
Return type:dict[str,dict[str]]
get_network_json_for_epoch(self, epoch)[source]
Parameters:epoch (int) – starting at 1
Return type:dict[str]
get_network_for_epoch(self, epoch, mask=None)[source]
Return type:Network.LayerNetwork
copy_params_from_old_network(self, new_network, old_network)[source]

:returns the remaining hidden layer names which exist only in the new network. :rtype: set[str]

get_train_param_args_for_epoch(self, epoch)[source]

:returns the kwargs for LayerNetwork.set_train_params, i.e. which params to train. :rtype: dict[str]

Return type:Pretrain | None

Will print out the different network topologies of the specified pretraining scheme.