TheanoUtil

TheanoUtil.time_batch_make_flat(val)[source]
Rtype val:theano.Variable
Return type:theano.Variable

Will flatten the first two dimensions and leave the others as is.

TheanoUtil.class_idx_seq_to_1_of_k(seq, num_classes, dtype='float32')[source]
Parameters:
  • seq (theano.Variable) – ndarray with indices
  • | theano.Variable num_classes (int) – number of classes
  • dtype (str) – eg “float32”
Return type:

theano.Variable

:returns ndarray with one added dimension of size num_classes. That is the one-hot-encoding. This function is like theano.tensor.extra_ops.to_one_hot but we can handle multiple dimensions.

TheanoUtil.tiled_eye(n1, n2, dtype='float32')[source]
TheanoUtil.opt_contiguous_on_gpu(x)[source]
TheanoUtil.windowed_batch(source, window)[source]
Parameters:
  • source (theano.TensorVariable) – 3d tensor of shape (n_time, n_batch, n_dim)
  • window (int|theano.Variable) – window size
Returns:

tensor of shape (n_time, n_batch, window * n_dim)

TheanoUtil.delta_batch(source, window)[source]
Parameters:
  • source (theano.TensorVariable) – 3d tensor of shape (n_time, n_batch, n_dim)
  • window (int|theano.Variable) – window size
Returns:

tensor of shape (n_time, n_batch, window * n_dim)

Similar as numpy.diff. Also called delta. TODO with conv op

TheanoUtil.context_batched(source, window)[source]

same as windowed_batch but with window center at the end of the window :param theano.TensorVariable source: 3d tensor of shape (n_time, n_batch, n_dim) :param int|theano.Variable window: window size :return: tensor of shape (n_time, n_batch, window * n_dim)

TheanoUtil.window_batch_timewise(t, b, w, full_index)[source]
TheanoUtil.slice_for_axis(axis, s)[source]
TheanoUtil.downsample(source, axis, factor, method='average')[source]
TheanoUtil.upsample(source, axis, factor, method='nearest-neighbor', target_axis_len=None)[source]
TheanoUtil.pad(source, axis, target_axis_len, pad_value=None)[source]
TheanoUtil.chunked_time_reverse(source, chunk_size)[source]
Parameters:
  • source – >=1d array (time,…)
  • chunk_size – int
Returns:

like source

Will not reverse the whole time-dim, but only every time-chunk. E.g. source=[0 1 2 3 4 5 6], chunk_size=3, returns [2 1 0 5 4 3 0]. (Padded with 0, recovers original size.)

TheanoUtil.try_register_canonicalize(f)[source]
TheanoUtil.try_register_gpu_opt(base_op_class)[source]
class TheanoUtil.GradDiscardOutOfBound(lower_bound, upper_bound)[source]
grad(args, g_outs)[source]
TheanoUtil.grad_discard_out_of_bound(x, lower_bound, upper_bound)[source]
TheanoUtil.gaussian_filter_1d(x, sigma, axis, window_radius=40)[source]

Filter 1d input with a Gaussian using mode nearest. x is expected to be 2D/3D of type (time,batch,…). Adapted via: https://github.com/Theano/Theano/issues/3793 Original Author: https://github.com/matthias-k

TheanoUtil.log_sum_exp(x, axis)[source]
TheanoUtil.max_filtered(x, axis, index)[source]
TheanoUtil.log_sum_exp_index(x, axis, index)[source]
TheanoUtil.global_softmax(z, index, mode)[source]
Parameters:
  • z (theano.Variable) – 3D array. time*batch*feature
  • index (theano.Variable) – 2D array, 0 or 1, time*batch
Return type:

theano.Variable

:returns 3D array. exp(z) / Z, where Z = sum(exp(z),axis=[0,2]) / z.shape[0].

TheanoUtil.show_global_softmax_stats(z)[source]
Parameters:z – numpy.ndarray or Theano Var (eval-able), 2D time*features
TheanoUtil.complex_elemwise_mult(a, b, axis=-1)[source]
TheanoUtil.complex_bound(a, axis=-1)[source]
TheanoUtil.complex_dot(a, b)[source]
TheanoUtil.indices_in_flatten_array(ndim, shape, *args)[source]

We expect that all args can be broadcasted together. So, if we have some array A with ndim&shape as given, A[args] would give us a subtensor. We return the indices so that A[args].flatten() and A.flatten()[indices] are the same.

TheanoUtil.circular_convolution(a, b)[source]
TheanoUtil.unroll_scan(fn, sequences=(), outputs_info=(), non_sequences=(), n_steps=None, go_backwards=False)[source]

Helper function to unroll for loops. Can be used to unroll theano.scan. The parameter names are identical to theano.scan, please refer to here for more information.

Note that this function does not support the truncate_gradient setting from theano.scan.

Code adapted from https://github.com/Lasagne/Lasagne. Thank you!

Parameters:
fn : function

Function that defines calculations at each step.

sequences : TensorVariable or list of TensorVariables

List of TensorVariable with sequence data. The function iterates over the first dimension of each TensorVariable.

outputs_info : list of TensorVariables

List of tensors specifying the initial values for each recurrent value.

non_sequences: list of TensorVariables

List of theano.shared variables that are used in the step function.

n_steps: int

Number of steps to unroll.

go_backwards: bool

If true the recursion starts at sequences[-1] and iterates backwards.

Returns:
Tuple of the form (outputs, updates).
outputs is a list of TensorVariables. Each element in the list gives the recurrent
values at each time step.
updates is an empty dict for now.
TheanoUtil.make_var_tuple(v)[source]
class TheanoUtil.Contiguous[source]
grad(inputs, output_grads)[source]
class TheanoUtil.DumpOp(filename, container=None, with_grad=True, parent=None, step=1)[source]
view_map = {0: [0]}[source]
make_node(x)[source]

Create a “apply” nodes for the inputs in that order.

perform(node, inputs, output_storage)[source]

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters:
node : Apply instance

Contains the symbolic inputs and outputs.

inputs : list

Sequence of inputs (immutable).

output_storage : list

List of mutable 1-element lists (do not change the length of these lists)

Raises:
MethodNotDefined

The subclass does not override this method.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

grad(inputs, output_grads)[source]
dump(x)[source]
get_full_filename()[source]
get_counter()[source]
inc_counter()[source]
TheanoUtil.softmax(z)[source]
TheanoUtil.layer_normalization(x, bias=None, scale=None, eps=1e-05)[source]

Layer Normalization, https://arxiv.org/abs/1607.06450 x is mean and variance normalized along its feature dimension. After that, we allow a bias and a rescale. This is supposed to be trainable. :param x: 3d tensor (time,batch,dim) (or any ndim, last dim is expected to be dim) :param bias: 1d tensor (dim) or None :param scale: 1d tensor (dim) or None

TheanoUtil.print_to_file(filename, x, argmax=None, sum=None, shape=False)[source]
TheanoUtil.self_similarity_cosine(x)[source]
Parameters:x – shape (T,D)

:returns cosine similarity matrix, shape (T,T), zeroed at diagonal and upper triangle