DCN V2

Reference:

Ruoxi Wang at al. “Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems.” in WWW 2021.

Reference code:

https://github.com/shenweichen/DeepCTR-Torch

class recbole.model.context_aware_recommender.dcnv2.DCNV2(config, dataset)

Bases: ContextRecommender

DCNV2 improves the cross network by extending the original weight vector to a matrix, significantly improves the expressiveness of DCN. It also introduces the MoE and low rank techniques to reduce time cost.

calculate_loss(interaction)

Calculate the training loss for a batch data.

Parameters:

interaction (Interaction) – Interaction class of the batch.

Returns:

Training loss, shape: []

Return type:

torch.Tensor

cross_network(x_0)

Cross network is composed of cross layers, with each layer having the following formula.

\[x_{l+1} = x_0 \odot (W_l x_l + b_l) + x_l\]

\(x_l\), \(x_{l+1}\) are column vectors denoting the outputs from the l -th and (l + 1)-th cross layers, respectively. \(W_l\), \(b_l\) are the weight and bias parameters of the l -th layer.

Parameters:

x_0 (torch.Tensor) – Embedding vectors of all features, input of cross network.

Returns:

output of cross network, [batch_size, num_feature_field * embedding_size]

Return type:

torch.Tensor

cross_network_mix(x_0)

Cross network part of DCN-mix, which add MoE and nonlinear transformation in low-rank space.

\[x_{l+1} = \sum_{i=1}^K G_i(x_l)E_i(x_l)+x_l\]

\[E_i(x_l) = x_0 \odot (U_l^i \dot g(C_l^i \dot g(V_L^{iT} x_l)) + b_l)\]

\(E_i\) and \(G_i\) represents the expert and gatings respectively, \(U_l\), \(C_l\), \(V_l\) stand for low-rank decomposition of weight matrix, \(g\) is the nonlinear activation function.

Parameters:

x_0 (torch.Tensor) – Embedding vectors of all features, input of cross network.

Returns:

output of mixed cross network, [batch_size, num_feature_field * embedding_size]

Return type:

torch.Tensor

forward(interaction)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

predict(interaction)

Predict the scores between users and items.

Parameters:

interaction (Interaction) – Interaction class of the batch.

Returns:

Predicted scores for given users and items, shape: [batch_size]

Return type:

torch.Tensor

training: bool