Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
RETURNN documentation
RETURNN documentation

Getting Started

  • Installation
  • Basic Usage
  • RETURNN frontend
  • Technological Overview
  • Tensor and Dim
  • Recurrency
  • Training
  • Native operations
  • RETURNN as Framework
  • Recent development of RETURNN
  • TensorFlow LSTM Benchmark
  • Frequently Asked Questions

User Guide

  • Network Structure
  • Dataset Input/Output
  • Recurrent Sub-Networks
  • Extending RETURNN

Reference

  • Configuration Parameters
    • behavior_version
    • General Settings
    • Training
    • Optimizer Settings
    • Pretraining
    • Model Loading
    • Generation and Search
    • Debugging
  • Datasets
    • Generic Datasets
    • Text Datasets
    • Audio Datasets
    • Dataset Combination
  • Layers / Modules
    • Basic Layers
    • Shape and Type Modification
    • Recurrent Layers
    • Recurrent Units
    • Attention Layers
    • Normalization Layers
    • Regularization Layers
    • Custom Layers
    • Utility Layers
    • Loss Functions
    • Softmax Layers
    • Generative Layers
    • Alignment Layers
  • Optimizer

Advanced Topics

  • Pre-Training / Dynamic Networks
  • Multi GPU training with PyTorch
  • Multi GPU training with TensorFlow
  • Debugging
  • Profiling (Python generic)
  • Profiling (PyTorch)
  • Profiling (TensorFlow)
  • Deterministic training
  • Test suite

Applications

  • Speech Recognition
  • Language Modelling
  • Machine Translation

Internals

  • API
    • returnn
    • returnn.__main__
    • returnn.__old_mod_loader__
    • returnn.__setup__
    • returnn.config
    • returnn.datasets
    • returnn.datasets.audio
    • returnn.datasets.basic
    • returnn.datasets.bundle_file
    • returnn.datasets.cached
    • returnn.datasets.cached2
    • returnn.datasets.distrib_files
    • returnn.datasets.generating
    • returnn.datasets.hdf
    • returnn.datasets.huggingface
    • returnn.datasets.lm
    • returnn.datasets.map
    • returnn.datasets.meta
    • returnn.datasets.multi_proc
    • returnn.datasets.normalization_data
    • returnn.datasets.numpy_dump
    • returnn.datasets.postprocessing
    • returnn.datasets.raw_wav
    • returnn.datasets.sprint
    • returnn.datasets.stereo
    • returnn.datasets.text_dict
    • returnn.datasets.util
    • returnn.datasets.util.feature_extraction
    • returnn.datasets.util.strings
    • returnn.datasets.util.vocabulary
    • returnn.engine
    • returnn.engine.base
    • returnn.engine.batch
    • returnn.extern
    • returnn.extern.HawkAaronWarpTransducer
    • returnn.extern.WarpRna
    • returnn.extern.WarpRna.__main__
    • returnn.extern.graph_editor
    • returnn.extern.graph_editor.edit
    • returnn.extern.graph_editor.reroute
    • returnn.extern.graph_editor.select
    • returnn.extern.graph_editor.subgraph
    • returnn.extern.graph_editor.transform
    • returnn.extern.graph_editor.util
    • returnn.extern.official_tf_resnet
    • returnn.extern.official_tf_resnet.resnet_model
    • returnn.forward_iface
    • returnn.frontend
    • returnn.frontend._backend
    • returnn.frontend._cache
    • returnn.frontend._native
    • returnn.frontend._numpy_backend
    • returnn.frontend._random_journal
    • returnn.frontend._utils
    • returnn.frontend.array_
    • returnn.frontend.assert_
    • returnn.frontend.attention
    • returnn.frontend.audio
    • returnn.frontend.audio.mel
    • returnn.frontend.audio.specaugment
    • returnn.frontend.backend
    • returnn.frontend.build_from_dict
    • returnn.frontend.cond
    • returnn.frontend.const
    • returnn.frontend.container
    • returnn.frontend.control_flow_ctx
    • returnn.frontend.conv
    • returnn.frontend.conversions
    • returnn.frontend.conversions.espnet_e_branchformer
    • returnn.frontend.conversions.hf_llama
    • returnn.frontend.conversions.torch_nn
    • returnn.frontend.decoder
    • returnn.frontend.decoder.transformer
    • returnn.frontend.device
    • returnn.frontend.dims
    • returnn.frontend.dropout
    • returnn.frontend.dtype
    • returnn.frontend.encoder
    • returnn.frontend.encoder.base
    • returnn.frontend.encoder.conformer
    • returnn.frontend.encoder.conformer_v2
    • returnn.frontend.encoder.e_branchformer
    • returnn.frontend.encoder.transformer
    • returnn.frontend.gradient
    • returnn.frontend.graph
    • returnn.frontend.hooks
    • returnn.frontend.init
    • returnn.frontend.label_smoothing
    • returnn.frontend.linear
    • returnn.frontend.loop
    • returnn.frontend.loss
    • returnn.frontend.math_
    • returnn.frontend.matmul
    • returnn.frontend.module
    • returnn.frontend.nested
    • returnn.frontend.normalization
    • returnn.frontend.parameter
    • returnn.frontend.parametrizations
    • returnn.frontend.parametrize
    • returnn.frontend.piecewise_linear
    • returnn.frontend.rand
    • returnn.frontend.rec
    • returnn.frontend.reduce
    • returnn.frontend.run_ctx
    • returnn.frontend.signal
    • returnn.frontend.state
    • returnn.frontend.stepwise_scheduler
    • returnn.frontend.tensor_array
    • returnn.frontend.types
    • returnn.import_
    • returnn.import_.common
    • returnn.import_.git
    • returnn.import_.import_
    • returnn.learning_rate_control
    • returnn.log
    • returnn.native_op
    • returnn.pretrain
    • returnn.sprint
    • returnn.sprint.cache
    • returnn.sprint.control
    • returnn.sprint.error_signals
    • returnn.sprint.extern_interface
    • returnn.sprint.interface
    • returnn.tensor
    • returnn.tensor._dim_extra
    • returnn.tensor._tensor_extra
    • returnn.tensor._tensor_mixin_base
    • returnn.tensor._tensor_op_overloads
    • returnn.tensor.control_flow_ctx
    • returnn.tensor.dim
    • returnn.tensor.marked_dim
    • returnn.tensor.tensor
    • returnn.tensor.tensor_dict
    • returnn.tensor.utils
    • returnn.tf
    • returnn.tf.compat
    • returnn.tf.data_pipeline
    • returnn.tf.distributed
    • returnn.tf.engine
    • returnn.tf.frontend_layers
    • returnn.tf.frontend_layers._backend
    • returnn.tf.frontend_layers._utils
    • returnn.tf.frontend_layers.cond
    • returnn.tf.frontend_layers.config_entry_points
    • returnn.tf.frontend_layers.debug_eager_mode
    • returnn.tf.frontend_layers.dims
    • returnn.tf.frontend_layers.layer
    • returnn.tf.frontend_layers.loop
    • returnn.tf.frontend_layers.make_layer
    • returnn.tf.frontend_layers.masked_computation
    • returnn.tf.frontend_layers.parameter_assign
    • returnn.tf.frontend_layers.prev_tensor_ref
    • returnn.tf.frontend_low_level
    • returnn.tf.frontend_low_level._backend
    • returnn.tf.horovod
    • returnn.tf.hyper_param_tuning
    • returnn.tf.layers
    • returnn.tf.layers.base
    • returnn.tf.layers.basic
    • returnn.tf.layers.rec
    • returnn.tf.layers.segmental_model
    • returnn.tf.layers.signal_processing
    • returnn.tf.layers.variable
    • returnn.tf.native_op
    • returnn.tf.network
    • returnn.tf.sprint
    • returnn.tf.updater
    • returnn.tf.util
    • returnn.tf.util.basic
    • returnn.tf.util.data
    • returnn.tf.util.gradient_checkpoint
    • returnn.tf.util.ken_lm
    • returnn.tf.util.open_fst
    • returnn.torch
    • returnn.torch.data
    • returnn.torch.data.extern_data
    • returnn.torch.data.pipeline
    • returnn.torch.data.queued_data_iter
    • returnn.torch.data.returnn_dataset_wrapper
    • returnn.torch.data.tensor_utils
    • returnn.torch.distributed
    • returnn.torch.engine
    • returnn.torch.frontend
    • returnn.torch.frontend._backend
    • returnn.torch.frontend._rand
    • returnn.torch.frontend.bridge
    • returnn.torch.frontend.compile_helper
    • returnn.torch.frontend.raw_ops
    • returnn.torch.optim
    • returnn.torch.optim.lion
    • returnn.torch.updater
    • returnn.torch.util
    • returnn.torch.util.array_
    • returnn.torch.util.assert_
    • returnn.torch.util.debug_inf_nan
    • returnn.torch.util.diagnose_gpu
    • returnn.torch.util.distributed
    • returnn.torch.util.exception_helper
    • returnn.torch.util.gradient_checkpoint
    • returnn.torch.util.module
    • returnn.torch.util.native_op
    • returnn.torch.util.native_op_code_compiler
    • returnn.torch.util.rope
    • returnn.torch.util.scaled_gradient
    • returnn.util
    • returnn.util.basic
    • returnn.util.better_exchook
    • returnn.util.bpe
    • returnn.util.collect_outputs_dict
    • returnn.util.cuda_env
    • returnn.util.debug
    • returnn.util.debug_helpers
    • returnn.util.file_cache
    • returnn.util.fsa
    • returnn.util.hot_reload
    • returnn.util.literal_py_to_pickle
    • returnn.util.lru_cache
    • returnn.util.math
    • returnn.util.multi_proc_manager_with_watchdog
    • returnn.util.multi_proc_non_daemonic_spawn
    • returnn.util.native_code_compiler
    • returnn.util.pprint
    • returnn.util.py_ext_mod_compiler
    • returnn.util.result_with_reason
    • returnn.util.sig_proc
    • returnn.util.task_system
    • returnn.util.train_proc_manager
    • returnn.util.watch_memory
  • TensorFlow Beam Search
Back to top

returnn.tf.layers¶

This package contains all the RETURNN TF layer implementations.

Next
returnn.tf.layers.base
Previous
returnn.tf.hyper_param_tuning
Copyright © 2014–2023, RETURNN contributors
Made with Sphinx and @pradyunsg's Furo