factory#

Factory for evaluation strategies.

exception MetricNotFoundError#

Bases: Exception

Raised when a metric is not found.

with_traceback()#

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

create(metric_id: str) → Metric#

Create a metric of a specific type.

Parameters:

metric_id – the metric to create.

Returns:

The metric.

Raises:

MetricNotFoundError – if the metric is unknown.

register(metric_id: str, metric_func: Metric) → None#

Register a new metric.

Parameters:

  • metric_id – the identifier of the metric.

  • metric_func – A function which takes ground-truth values, predicted values, and returns their distance.

unregister(metric_id: str) → None#

Unregister a metric.

Parameters:

metric_id – the metric type to unregister.