recbole.evaluator.abstract_metric

class recbole.evaluator.base_metric.AbstractMetric(config)[source]

Bases: object

AbstractMetric is the base object of all metrics. If you want to

implement a metric, you should inherit this class.

Parameters

config (Config) – the config of evaluator.

calculate_metric(dataobject)[source]

Get the dictionary of a metric.

Parameters

dataobject (DataStruct) – it contains all the information needed to calculate metrics.

Returns

such as {'metric@10': 3153, 'metric@20': 0.3824}

Return type

dict

smaller = False
class recbole.evaluator.base_metric.LossMetric(config)[source]

Bases: recbole.evaluator.base_metric.AbstractMetric

LossMetric is a base object of loss based metrics and AUC. If you want to implement an loss based metric, you can inherit this class.

Parameters

config (Config) – The config of evaluator.

metric_info(preds, trues)[source]

Calculate the value of the metric.

Parameters
  • preds (numpy.ndarray) – the scores predicted by model, a one-dimensional vector.

  • trues (numpy.ndarray) – the label of items, which has the same shape as preds.

Returns

The value of the metric.

Return type

float

metric_need = ['rec.score', 'data.label']
metric_type = 2
output_metric(metric, dataobject)[source]
used_info(dataobject)[source]

Get scores that model predicted and the ground truth.

class recbole.evaluator.base_metric.TopkMetric(config)[source]

Bases: recbole.evaluator.base_metric.AbstractMetric

TopkMetric is a base object of top-k metrics. If you want to implement an top-k metric, you can inherit this class.

Parameters

config (Config) – The config of evaluator.

metric_info(pos_index, pos_len=None)[source]

Calculate the value of the metric.

Parameters
  • pos_index (numpy.ndarray) – a bool matrix, shape of n_users * max(topk). The item with the (j+1)-th highest score of i-th user is positive if pos_index[i][j] == True and negative otherwise.

  • pos_len (numpy.ndarray) – a vector representing the number of positive items per user, shape of (n_users,).

Returns

metrics for each user, including values from metric@1 to metric@max(self.topk).

Return type

numpy.ndarray

metric_need = ['rec.topk']
metric_type = 1
topk_result(metric, value)[source]

Match the metric value to the k and put them in dictionary form.

Parameters
  • metric (str) – the name of calculated metric.

  • value (numpy.ndarray) – metrics for each user, including values from metric@1 to metric@max(self.topk).

Returns

metric values required in the configuration.

Return type

dict

used_info(dataobject)[source]

Get the bool matrix indicating whether the corresponding item is positive and number of positive items for each user.