recbole.evaluator.loss_evaluator

class recbole.evaluator.loss_evaluator.LossEvaluator(config)

Bases: recbole.evaluator.abstract_evaluator.AbstractEvaluator

Loss Evaluator is mainly used in rating prediction and click through rate prediction. Now, we support four loss metrics which contain ‘AUC’, ‘RMSE’, ‘MAE’, ‘LOGLOSS’.

Note

The metrics used do not calculate group-based metrics which considers the metrics scores averaged across users. They are also not limited to k. Instead, they calculate the scores on the entire prediction results regardless the users.

collect(interaction, pred_scores)

collect the loss intermediate result of one batch, this function mainly implements concatenating preds and trues. It is called at the end of each batch

Parameters

• interaction (Interaction) – AbstractEvaluator of the batch

• pred_scores (tensor) – the tensor of model output with a size of (N, )

Returns

a batch of socres with a size of (N, 2)

Return type

tensor

evaluate(batch_matrix_list, *args)

calculate the metrics of all batches. It is called at the end of each epoch

Parameters

batch_matrix_list (list) – the results of all batches

Returns

such as {‘AUC’: 0.83}

Return type

dict

metrics_info(trues, preds)

get metrics result

Parameters

• trues (np.ndarray) – the true scores’ list

• preds (np.ndarray) – the predict scores’ list

Returns

a list of metrics result

Return type

list