tfaip.lav

LAV

Implementation of LAV

class tfaip.lav.lav.MetricsAccumulator

Bases: object

Computation of the running weighted sum (=weighted mean) of a dictionary of float (here the metrics of the scenario) Call final() to access the mean.

running_sum: The running weighted sum of all values

running_weight: The sum of all weights

__init__()

accumulate_dict_sum(new_values: Dict[str, float], sample_weights: Dict[str, float])

final()

class tfaip.lav.lav.InplaceCallbackList(callbacks=None, add_history=False, add_progbar=False, model=None, **params)

Bases: keras.callbacks.CallbackList

Callback list that does not create new log objects

This is required in Tensorflow 2.5.x to support adding and removing of log entries

class tfaip.lav.lav.EvaluationCallback(evaluator: tfaip.evaluator.evaluator.EvaluatorBase, data_pipeline, callbacks: List[tfaip.lav.callbacks.lav_callback.LAVCallback])

Bases: tensorflow.python.keras.callbacks.Callback

Callback to run the post-proc pipeline and call the evaluator run post proc pipeline in a separate thread so that it is instantiated only once the EvaluationCallback handles writing to post proc queue, reading the post proc data back again evaluating the result and writing it to the logs

__init__(evaluator: tfaip.evaluator.evaluator.EvaluatorBase, data_pipeline, callbacks: List[tfaip.lav.callbacks.lav_callback.LAVCallback])

on_test_batch_end(batch, logs=None)

Called at the end of a batch in evaluate methods.

Also called at the end of a validation batch in the fit methods, if validation data is provided.

Subclasses should override for any actions to run.

Note that if the steps_per_execution argument to compile in tf.keras.Model is set to N, this method will only be called every N batches.

Parameters

batch – Integer, index of batch within the current epoch.
logs – Dict. Aggregated metric results up until this batch.

on_test_end(logs=None)

Called at the end of evaluation or validation.

Subclasses should override for any actions to run.

Parameters: logs – Dict. Currently the output of the last call to on_test_batch_end() is passed to this argument for this method but that may change in the future.

class tfaip.lav.lav.LAV(params: tfaip.lav.params.LAVParams, scenario_params: tfaip.scenario.scenariobaseparams.ScenarioBaseParams, **kwargs)

Bases: abc.ABC

This is the base Class for Loading And Validation (LAV) a scenario. By default, LAV is independent of the scenario and should therefore not be overridden (but it is possible though in Meta of a Scenario).

LAV will open an exported (or the best) model, create the metrics of the actual scenario. Then predict is called on the model on the DataBase validation(!) set and fed to the metric to obtain results (MetricsAccumulator). During the prediction a models print_evaluate method is called to print informative text.

Override _custom_callbacks to “inject” logic for processing samples during LAV (e.g. saving or printing).

classmethod params_cls() → Type[tfaip.lav.params.LAVParams]

__init__(params: tfaip.lav.params.LAVParams, scenario_params: tfaip.scenario.scenariobaseparams.ScenarioBaseParams, **kwargs)

property params: tfaip.lav.params.LAVParams

run(**kwargs)

extract_dump_data(sample: tfaip.data.pipeline.definitions.Sample)

MultiLAV

Implementation of MultiLAV

class tfaip.lav.multilav.MultiLAV(params: tfaip.lav.params.LAVParams, scenario_params: tfaip.scenario.scenariobaseparams.ScenarioBaseParams, predictor_params: tfaip.predict.params.PredictorParams, **kwargs)

Bases: abc.ABC

Similar to LAV but supports multiple models and voting. Drawback: does not use metrics defined in model (EvaluatorBase and Voter must handle everything)

Therefore the implementation differs from the classical LAV, a MultiPredictor is created whose outputs are passed to the evaluator.

classmethod params_cls() → Type[tfaip.lav.params.LAVParams]

__init__(params: tfaip.lav.params.LAVParams, scenario_params: tfaip.scenario.scenariobaseparams.ScenarioBaseParams, predictor_params: tfaip.predict.params.PredictorParams, **kwargs)

run(**kwargs)

extract_dump_data(sample: tfaip.data.pipeline.definitions.Sample)

LAVParams

Implementation of the LAVParams

class tfaip.lav.params.LAVParams(pipeline: tfaip.data.databaseparams.DataPipelineParams = <factory>, model_path: Optional[Union[str, List[str]]] = None, device: tfaip.device.device_config.DeviceConfigParams = <factory>, print_samples: bool = False, silent: bool = False, store_results: bool = True)

Bases: object

Parameters for LAV

pipeline: tfaip.data.databaseparams.DataPipelineParams

model_path: Union[str, List[str]] = None

device: tfaip.device.device_config.DeviceConfigParams

print_samples: bool = False

silent: bool = False

store_results: bool = True

__init__(pipeline: tfaip.data.databaseparams.DataPipelineParams = <factory>, model_path: Optional[Union[str, List[str]]] = None, device: tfaip.device.device_config.DeviceConfigParams = <factory>, print_samples: bool = False, silent: bool = False, store_results: bool = True) → None

classmethod from_dict(kvs: Optional[Union[dict, list, str, int, float, bool]], *, infer_missing=False) → dataclasses_json.api.A

classmethod from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) → dataclasses_json.api.A

classmethod schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) → dataclasses_json.mm.SchemaF[dataclasses_json.mm.A]

to_dict(encode_json=False, include_cls=True) → Dict[str, Optional[Union[dict, list, str, int, float, bool]]]

to_json(*, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Optional[Union[int, str]] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) → str

Callbacks

Implementation of a general LAVCallback

class tfaip.lav.callbacks.lav_callback.LAVCallback

Bases: abc.ABC

Base class of a callback that can be added to LAV

__init__()

setup(data_generator_params: tfaip.data.databaseparams.DataGeneratorParams, lav: LAV, data: DataBase, model: ModelBase)

on_lav_start()

on_sample_end(sample: tfaip.data.pipeline.definitions.Sample)

on_lav_end(result)

Definition of the DumpResultsCallback using in LAV

class tfaip.lav.callbacks.dump_results.DumpResultsCallback(filepath: str)

Bases: tfaip.lav.callbacks.lav_callback.LAVCallback

Callback that dumps all results to a file using pickle.

__init__(filepath: str)

on_sample_end(sample)

on_lav_end(result)