DIEN

Reference:

Guorui Zhou et al. “Deep Interest Evolution Network for Click-Through Rate Prediction” in AAAI 2019

Reference code:

• https://github.com/mouna99/dien

• https://github.com/shenweichen/DeepCTR-Torch/

class recbole.model.sequential_recommender.dien.AGRUCell(input_size, hidden_size, bias=True)

Bases: torch.nn.modules.module.Module

Attention based GRU (AGRU). AGRU uses the attention score to replace the update gate of GRU, and changes the

hidden state directly.

Formally:

..math: {h}_{t}^{prime}=left(1-a_{t}

ight) * {h}_{t-1}^{prime}+a_{t} * ilde{{h}}_{t}^{prime}

\({h}_{t}^{\prime}\), \(h_{t-1}^{\prime}\), \({h}_{t-1}^{\prime}\), :math: ` ilde{{h}}_{t}^{prime}` are the hidden state of AGRU

forward(input, hidden_output, att_score)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class recbole.model.sequential_recommender.dien.AUGRUCell(input_size, hidden_size, bias=True)

Bases: torch.nn.modules.module.Module

Effect of GRU with attentional update gate (AUGRU). AUGRU combines attention mechanism and GRU seamlessly.

Formally:

..math: ilde{{u}}_{t}^{prime}=a_{t} * {u}_{t}^{prime}

{h}_{t}^{prime}=left(1- ilde{{u}}_{t}^{prime}

ight) circ {h}_{t-1}^{prime}+ ilde{{u}}_{t}^{prime} circ ilde{{h}}_{t}^{prime}

forward(input, hidden_output, att_score)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class recbole.model.sequential_recommender.dien.DIEN(config, dataset)

Bases: recbole.model.abstract_recommender.SequentialRecommender

DIEN has an interest extractor layer to capture temporal interests from history behavior sequence,and an interest evolving layer to capture interest evolving process that is relative to the target item. At interest evolving layer, attention mechanism is embedded intothe sequential structure novelly, and the effects of relative interests are strengthened during interest evolution.

calculate_loss(interaction)

Calculate the training loss for a batch data.

Parameters

interaction (Interaction) – Interaction class of the batch.

Returns

Training loss, shape: []

Return type

torch.Tensor

forward(user, item_seq, neg_item_seq, item_seq_len, next_items)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

input_type = 1

predict(interaction)

Predict the scores between users and items.

Parameters

interaction (Interaction) – Interaction class of the batch.

Returns

Predicted scores for given users and items, shape: [batch_size]

Return type

torch.Tensor

training: bool

class recbole.model.sequential_recommender.dien.DynamicRNN(input_size, hidden_size, bias=True, gru='AGRU')

Bases: torch.nn.modules.module.Module

forward(input, att_scores=None, hidden_output=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class recbole.model.sequential_recommender.dien.InterestEvolvingLayer(mask_mat, input_size, rnn_hidden_size, att_hidden_size=(80, 40), activation='sigmoid', softmax_stag=True, gru='GRU')

Bases: torch.nn.modules.module.Module

As the joint influence from external environment and internal cognition, different kinds of user interests are evolving over time. Interest Evolving Layer can capture interest evolving process that is relative to the target item.

final_output(outputs, keys_length)

get the last effective value in the interest evolution sequence :param outputs: the output of DynamicRNN after pad_packed_sequence :type outputs: torch.Tensor :param keys_length: the true length of the user history sequence :type keys_length: torch.Tensor

Returns

The user’s CTR for the next item

Return type

torch.Tensor

forward(queries, keys, keys_length)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class recbole.model.sequential_recommender.dien.InterestExtractorNetwork(input_size, hidden_size, mlp_size)

Bases: torch.nn.modules.module.Module

In e-commerce system, user behavior is the carrier of latent interest, and interest will change after user takes one behavior. At the interest extractor layer, DIEN extracts series of interest states from sequential user behaviors.

auxiliary_loss(h_states, click_seq, noclick_seq, keys_length)

Computes the auxiliary loss

Formally: ..math: L_{a u x}= frac{1}{N}(sum_{i=1}^{N} sum_{t} log sigma(mathbf{h}_{t}^{i}, mathbf{e}_{b}^{i}[t+1])

• log (1-sigma(mathbf{h}_{t}^{i}, hat{mathbf{e}}_{b}^{i}[t+1])))

Parameters

• h_states (torch.Tensor) – The output of GRUs’ hidden layer, [batch_size, history_length - 1, embedding,size].

• click_seq (torch.Tensor) – The sequence that users consumed, [batch_size, history_length - 1, embedding,size].

• noclick_seq – The sequence that users did not consume, [batch_size, history_length - 1, embedding_size].

forward(keys, keys_length, neg_keys=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool