recbole.evaluator.abstract_metric

class recbole.evaluator.base_metric.AbstractMetric(config)

Bases: object

AbstractMetric is the base object of all metrics. If you want to

implement a metric, you should inherit this class.

Parameters

config (Config) – the config of evaluator.

calculate_metric(dataobject)

Get the dictionary of a metric.

Parameters

dataobject (DataStruct) – it contains all the information needed to calculate metrics.

Returns

such as {'metric@10': 3153, 'metric@20': 0.3824}

Return type

dict

smaller = False

class recbole.evaluator.base_metric.LossMetric(config)

Bases: recbole.evaluator.base_metric.AbstractMetric

LossMetric is a base object of loss based metrics and AUC. If you want to implement an loss based metric, you can inherit this class.

Parameters

config (Config) – The config of evaluator.

metric_info(preds, trues)

Calculate the value of the metric.

Parameters

• preds (numpy.ndarray) – the scores predicted by model, a one-dimensional vector.

• trues (numpy.ndarray) – the label of items, which has the same shape as preds.

Returns

The value of the metric.

Return type

float

metric_need = ['rec.score', 'data.label']

metric_type = 2

output_metric(metric, dataobject)

used_info(dataobject)

Get scores that model predicted and the ground truth.

class recbole.evaluator.base_metric.TopkMetric(config)

Bases: recbole.evaluator.base_metric.AbstractMetric

TopkMetric is a base object of top-k metrics. If you want to implement an top-k metric, you can inherit this class.

Parameters

config (Config) – The config of evaluator.

metric_info(pos_index, pos_len=None)

Calculate the value of the metric.

Parameters

• pos_index (numpy.ndarray) – a bool matrix, shape of n_users * max(topk). The item with the (j+1)-th highest score of i-th user is positive if pos_index[i][j] == True and negative otherwise.

• pos_len (numpy.ndarray) – a vector representing the number of positive items per user, shape of (n_users,).

Returns

metrics for each user, including values from metric@1 to metric@max(self.topk).

Return type

numpy.ndarray

metric_need = ['rec.topk']

metric_type = 1

topk_result(metric, value)

Match the metric value to the k and put them in dictionary form.

Parameters

• metric (str) – the name of calculated metric.

• value (numpy.ndarray) – metrics for each user, including values from metric@1 to metric@max(self.topk).

Returns

metric values required in the configuration.

Return type

dict

used_info(dataobject)

Get the bool matrix indicating whether the corresponding item is positive and number of positive items for each user.