returnn.frontend.rec

Provides the LSTM module.

class returnn.frontend.rec.LSTM(in_dim: Dim, out_dim: Dim, *, with_bias: bool = True)

LSTM module.

Code to initialize the LSTM module.

default_initial_state(*, batch_dims: Sequence[Dim]) → LstmState

initial state

class returnn.frontend.rec.LstmState(*_args, h: Tensor | None = None, c: Tensor | None = None)

LSTM state

class returnn.frontend.rec.ZoneoutLSTM(in_dim: Dim, out_dim: Dim, *, with_bias: bool = True, zoneout_factor_cell: float = 0.0, zoneout_factor_output: float = 0.0, use_zoneout_output: bool = True, forget_bias: float = 0.0, parts_order: str = 'ifco')

Zoneout LSTM module.

Parameters:

• in_dim

• out_dim

• with_bias

• zoneout_factor_cell – 0.0 is disabled. reasonable is 0.15.

• zoneout_factor_output – 0.0 is disabled. reasonable is 0.05.

• use_zoneout_output – True is like the original paper. False is like older RETURNN versions.

• forget_bias – 1.0 is default in RETURNN/TF ZoneoutLSTM. 0.0 is default in LSTM, or RETURNN NativeLSTM, PyTorch LSTM, etc.

• parts_order – i: input gate. f: forget gate. o: output gate. c|g|j: input. icfo: like RETURNN/TF ZoneoutLSTM. ifco: PyTorch (cuDNN) weights, standard for LSTM. cifo: RETURNN NativeLstm2 weights.