Generic Datasets#

HDF Dataset#

class HDFDataset.HDFDataset(files=None, use_cache_manager=False, **kwargs)[source]#

Bases: CachedDataset

Dataset based on HDF files. This was the main original dataset format of RETURNN.

Parameters:

  • files (None|list[str]) –

  • use_cache_manager (bool) – uses Util.cf() for files

Next Gen HDF Dataset#

class HDFDataset.NextGenHDFDataset(input_stream_name, files=None, **kwargs)[source]#

Bases: CachedDataset2

Another separate dataset which uses HDF files to store the data.

Parameters:

  • input_stream_name (str) –

  • files (None|list[str]) –