returnn.frontend.stepwise_scheduler

Stepwise scheduler, e.g. for learning rate or other hyperparameters.

All these modules will accept any args/kwargs but leave them unused, and instead uses get_run_ctx() to get the current train step from the current run context.

class returnn.frontend.stepwise_scheduler.PiecewiseLinearStepwiseScheduler(points: Dict[int | float, float | Tensor])[source]

Piecewise linear scheduler based on the current global train step.

Example:

scheduler = PiecewiseLinearStepwiseScheduler(
    {0: 1.0, 10000: 0.1, 20000: 0.01}
)

This will start with 1.0, and then linearly decay to 0.1 at step 10000, and then to 0.01 at step 20000.

Parameters:

points – dict of key -> value pairs.