EngineTask

class EngineTask.TaskThread(task, network, devices, data, batches, eval_batch_size=0, start_batch=0, share_batches=False, reduction_rate=1.0, report_prefix=None, exclude=None, epoch=None)[source]
Parameters:report_prefix (str) – such as epoch or so. only for reporting
batch_idx = None[source]
Type:int | None
device_crash_batch = None[source]
Type:int | None
assign_dev_data(device, batches)[source]
maybe_wait_for_batches(device, batches)[source]
allocate_devices(selected_devices=None)[source]

Sets the device data, i.e. the next batches, via self.batches. This calls Dataset.load_seqs() to get the data. This sets:

device.targets device.ctc_targets device.tags device.index
Return type:list[list[EngineBatch.Batch]]

:returns list of batches per device

prepare_device_for_batch(device)[source]
get_device_prepare_args()[source]
evaluate(batchess, results, result_format, num_frames)[source]
Parameters:
  • batchess (list[list[EngineBatch.Batch]]) – batches per device
  • results (list[list[numpy.ndarray]]) – results per device
  • result_format (list[str]|None) – describes what we have in a result list

:returns some score or None :rtype: dict[str] | None

initialize()[source]

Called at the beginning of an epoch.

reduce(num_frames)[source]
epoch_norm_factor_for_result(key)[source]
finalize()[source]

Called at the end of an epoch.

class DeviceBatchRun(parent, devices)[source]
eval_info = None[source]
Type:dict[str] | None
allocate()[source]
finish()[source]

:returns whether everything is fine.

run()[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

stop()[source]
device_run()[source]
device_collect_results()[source]
device_mem_usage_str(devices)[source]
Return type:str | None
print_process()[source]
run()[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

run_inner()[source]
exception EngineTask.ModelBrokenError(msg, batches)[source]

We got a nan/inf at the result somewhere. This means that something is broken.

class EngineTask.TrainTaskThread(network, devices, data, batches, learning_rate, updater, seq_train_parallel=None, **kwargs)[source]
initialize()[source]

Called at the beginning of an epoch.

prepare_device_for_batch(device)[source]
get_device_prepare_args()[source]
maybe_wait_for_batches(device, batches)[source]
save_ctc_priors(filename, epoch_str)[source]
class CopyManager(devices)[source]
class CopyThread(device, network, copy_to_device)[source]
run()[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

copy_to_device(network)[source]
copy_from_device()[source]
reduce(num_frames)[source]
finalize()[source]

Called at the end of an epoch.

class EngineTask.EvalTaskThread(network, devices, data, batches, **kwargs)[source]
initialize()[source]

Called at the beginning of an epoch.

class EngineTask.ForwardTaskThread(network, devices, data, batches, eval_batch_size=0)[source]
evaluate(batchess, results, result_format, num_frames)[source]
Parameters:
  • batchess (list[list[EngineBatch.Batch]]) – batches per device
  • results (list[list[numpy.ndarray]]) – results per device
  • result_format (list[str]|None) – describes what we have in a result list

:returns some score or None :rtype: dict[str] | None

class EngineTask.HDFForwardTaskThread(network, devices, data, batches, cache, compression='none')[source]
initialize()[source]

Called at the beginning of an epoch.

finalize()[source]

Called at the end of an epoch.

evaluate(batchess, results, result_format, num_frames)[source]
Parameters:
  • batchess (list[list[Batch]]) – batches per device
  • results (list[list[numpy.ndarray]]) – results per device
  • result_format (list[str]|None) – describes what we have in a result list

:returns some score or None :rtype: dict[str] | None

class EngineTask.ClassificationTaskThread(network, devices, data, batches)[source]
evaluate(batchess, results, result_format, num_frames)[source]
Parameters:
  • batchess (list[list[EngineBatch.Batch]]) – batches per device
  • results (list[list[numpy.ndarray]]) – results per device
  • result_format (list[str]|None) – describes what we have in a result list

:returns some score or None :rtype: dict[str] | None

class EngineTask.PriorEstimationTaskThread(network, devices, data, batches, priori_file, target, extract_type)[source]
evaluate(batchess, results, result_format, num_frames)[source]
Parameters:
  • batchess (list[list[EngineBatch.Batch]]) – batches per device
  • results (list[list[numpy.ndarray]]) – results per device
  • result_format (list[str]|None) – describes what we have in a result list

:returns some score or None :rtype: dict[str] | None

finalize()[source]

Called at the end of an epoch.