recbole.evaluator.abstract_evaluator

class recbole.evaluator.abstract_evaluator.BaseEvaluator(config, metrics)[source]

Bases: object

BaseEvaluator is an object which supports the evaluation of the model. It is called by Trainer.

Note

If you want to inherit this class and implement your own evaluator class, you must implement the following functions.

Parameters

config (Config) – The config of evaluator.

collect(*args)[source]

get the intermediate results for each batch, it is called at the end of each batch

evaluate(*args)[source]

calculate the metrics of all batches, it is called at the end of each epoch

class recbole.evaluator.abstract_evaluator.GroupedEvaluator(config, metrics)[source]

Bases: recbole.evaluator.abstract_evaluator.BaseEvaluator

GroupedEvaluator is an object which supports the evaluation of the model.

Note

If you want to implement a new group-based metric, you may need to inherit this class

full_sort_collect(scores_tensor, user_len_list)[source]

it is called when evaluation sample distribution is full.

get_score_matrix(scores_tensor, user_len_list)[source]

get score matrix.

Parameters

  • scores_tensor (tensor) – the tensor of model output with size of (N, )

  • user_len_list (list) – number of all items

sample_collect(scores_tensor, user_len_list)[source]

padding scores_tensor. It is called when evaluation sample distribution is uniform or popularity.

class recbole.evaluator.abstract_evaluator.IndividualEvaluator(config, metrics)[source]

Bases: recbole.evaluator.abstract_evaluator.BaseEvaluator

IndividualEvaluator is an object which supports the evaluation of the model.

Note

If you want to implement a new non-group-based metric, you may need to inherit this class

full_sort_collect(true_scores, pred_scores)[source]

it is called when evaluation sample distribution is full.

get_score_matrix(true_scores, pred_scores)[source]

get score matrix

Parameters

  • true_scores (tensor) – the label of predicted items

  • pred_scores (tensor) – the tensor of model output with a size of (N, )

sample_collect(true_scores, pred_scores)[source]

It is called when evaluation sample distribution is uniform or popularity.