recbole.evaluator.topk_evaluator

class recbole.evaluator.topk_evaluator.TopKEvaluator(config)

Bases: recbole.evaluator.abstract_evaluator.AbstractEvaluator

TopK Evaluator is mainly used in ranking tasks. Now, we support six topk metrics which contain ‘Hit’, ‘Recall’, ‘MRR’, ‘Precision’, ‘NDCG’, ‘MAP’.

Note

The metrics used calculate group-based metrics which considers the metrics scores averaged across users. Some of them are also limited to k.

collect(interaction, scores_tensor, full=False)

collect the topk intermediate result of one batch, this function mainly implements padding and TopK finding. It is called at the end of each batch

Parameters

• interaction (Interaction) – AbstractEvaluator of the batch

• scores_tensor (tensor) – the tensor of model output with size of (N, )

• full (bool, optional) – whether it is full sort. Default: False.

evaluate(batch_matrix_list, eval_data)

calculate the metrics of all batches. It is called at the end of each epoch

Parameters

• batch_matrix_list (list) – the results of all batches

• eval_data (Dataset) – the class of test data

Returns

such as {'Hit@20': 0.3824, 'Recall@20': 0.0527, 'Hit@10': 0.3153, 'Recall@10': 0.0329}

Return type

dict

metrics_info(pos_idx, pos_len)

get metrics result

Parameters

• pos_idx (np.ndarray) – the bool index of all users’ topk items that indicating the postive items are topk items or not

• pos_len (list) – the length of all users’ postivite items

Returns

a list of matrix which record the results from 1 to max(topk)

Return type

list