`returnn.frontend.rand`¶

Random number generator utilities.

Note on the global seed:

In the TF engine, we have this in _init_network:

tf_random_seed = 42
net_random_seed = epoch
if self.config.opt_typed_value("random_seed", None):
    seed = self.config.int("random_seed", None)
    net_random_seed = (epoch * 3 + seed * 5 + 7) % (2**31)
    tf_random_seed = (net_random_seed * 2 + 3) % (2**31)
tf_compat.v1.set_random_seed(tf_random_seed)

_init_network is only called in the beginning and then only when needed (e.g. different model due to pretrain).

The net_random_seed is used inside the TFNetwork for:

self.random = numpy.random.RandomState(rnd_seed)

This in turns is used for param init in the TF layers, like:

fwd_weights_initializer = get_initializer(
    forward_weights_init, seed=self.network.random.randint(2**31), eval_local_ns={"layer": self}
)

It is also used by the RandomLayer when static=True:

# static is True
if seed is None:
    seed = self.network.random.randint(2**31, size=[32], dtype="uint32")

So, it means, with static=True, the random numbers are always the same in each execution step. To reflect that in an eager-based backend, we need to reset the static-random-state in the beginning of each step.

returnn.frontend.rand.set_random_seed(seed: int)[source]¶

Call this at the beginning of the program (after the RF backend was selected), or when the model and computation graph is supposed to be reinitialized.

This initializes the random state of the backend and also the step-based random state.

This is not expected to be called after each epoch or step.

Parameters:: seed – should depend on epoch or step

returnn.frontend.rand.get_random_state() → Dict[str, bytes][source]¶

Returns:: current random state, for serialization, to be able to restore it later

returnn.frontend.rand.set_random_state(state: Dict[str, bytes])[source]¶

Recovers the random state.

There are many potential cases where we cannot recover the state (e.g. different backend version, different hardware, …), In this case, a run without interruption is not the same as a run with interruption.

We still assume that set_random_seed() was called before in any case.

Parameters:: state – as returned by get_random_state()

returnn.frontend.rand.reset_step_random_state()[source]¶: When static=True is used in random(), the random state is reset to the beginning of the step. So this should be called in the beginning of each step. Also see the module docstring.

returnn.frontend.rand.get_static_step_based_seed(*, size=None) → int | ndarray[source]¶

Returns:: from the static step-based random state, get a seed

Generates random numbers from uniform or normal or truncated normal distribution.

There will be no gradients to mean, stddev, bound, minval, maxval!

In case of TensorFlow: This uses the TensorFlow stateless random ops internally, i.e. all the state handling is explicit. The state var can be explicitly provided and initialized via RandomStateInitLayer, or when not provided it will be automatically created.

There are two possible distinct use cases:

For any randomness in the model, e.g. dropout. So each session.run step will produce a new random number and advance the random state.
To initialize parameters via the config, using VariableLayer with the init_by_layer option. This will only be called once when initializing the parameters. For this use case, we do not want to keep a random state var. You can just pass static=False. Alternatively you could also pass the output of a RandomStateInitLayer as state.

Parameters:

dims
dtype
device
sparse_dim
feature_dim
distribution (str) – “uniform”, “normal” or “truncated_normal”
mean (int|float|Tensor|None)
stddev (int|float|Tensor|None)
bound (int|float|Tensor|None) – for uniform, defining the range [-bound, bound)
minval (int|float|Tensor|None) – for uniform
maxval (int|float|Tensor|None) – for uniform
seed (int|list[int]|numpy.ndarray|None) – If not given, uses self.network.random.randint, i.e. then it is controlled by the global seed setting, and every layer would get its own seed. If you specify it explicitly, make sure every RandomLayer uses a different seed, otherwise you would get the same random numbers everywhere.
algorithm (str|tf.random.Algorithm|None) – see RandomStateInitLayer
explicit_state (Tensor|None) – You can pass the state explicitly here. If not given, will be created automatically, and updated automatically. You could pass a VariableLayer with initial value via RandomStateInitLayer, or directly a RandomStateInitLayer. If auto_update_state is True, it must be a variable, and every time a new random number is created, this variable is updated. Otherwise (default), it will not be updated automatically.
auto_update_state (bool|None) – only used when you pass an explicit state
static (bool|None) – if no state at all should be used. it just relies on the seed then.
out – if given, will directly write into it, if possible by backend

Returns:

random values

returnn.frontend.rand.random_uniform(dims: Sequence[Dim], *, dtype: str | None = None, device: str | None = None, sparse_dim: Dim | None = None, feature_dim: Dim | None = None, minval: int | float | Tensor = 0, maxval: int | float | Tensor = 1, seed: int | Sequence[int] | ndarray | None = None, algorithm: str | None = None, explicit_state: Tensor | None = None, auto_update_state: bool | None = None, static: bool | None = None, out: Tensor | None = None)[source]¶: See random(). random() with distribution="uniform".

returnn.frontend.rand.random_normal(dims: Sequence[Dim], *, dtype: str | None = None, device: str | None = None, sparse_dim: Dim | None = None, feature_dim: Dim | None = None, mean: int | float | Tensor | None = 0.0, stddev: int | float | Tensor | None = 1.0, seed: int | Sequence[int] | ndarray | None = None, algorithm: str | None = None, explicit_state: Tensor | None = None, auto_update_state: bool | None = None, static: bool | None = None, out: Tensor | None = None)[source]¶: See random(). random() with distribution="normal".

returnn.frontend.rand.random_truncated_normal(dims: Sequence[Dim], *, dtype: str | None = None, device: str | None = None, sparse_dim: Dim | None = None, feature_dim: Dim | None = None, mean: int | float | Tensor | None = 0.0, stddev: int | float | Tensor | None = 1.0, minval: int | float | Tensor | None = None, maxval: int | float | Tensor | None = None, seed: int | Sequence[int] | ndarray | None = None, algorithm: str | None = None, explicit_state: Tensor | None = None, auto_update_state: bool | None = None, static: bool | None = None, out: Tensor | None = None)[source]¶: See random(). random() with distribution="truncated_normal".

returnn.frontend.rand¶

`returnn.frontend.rand`¶