Paper → implementation
See this paper in code.
Community-written snippets that make the paper’s ideas concrete. These are not official implementations from the paper’s authors.
From a batch of histories to predictions
One training example contains an entire history plus one candidate. items [B, N, D] holds B examples; each example's history has N positions, and each position holds one item vector of D features. targets [B, D] holds one candidate per example. The scorer produces weights [B, N], pooling produces interest [B, D], and the prediction head produces B click logits. The main section runs all B examples together with tiny hand-written vectors and no padding.
16 17if __name__ == "__main__":18 torch.manual_seed(0)19 items = torch.tensor([[[1., 0.], [0., 2.]],20 [[0., 1.], [2., 0.]]]) # [B=2, N=2, D=2]21 targets = torch.tensor([[1., 0.5], [0.5, 1.]]) # [B, D]22 activation = LocalActivation(dim=2)Build a target-aware embedding module
Move from supplied feature vectors to item IDs. TargetAwareEmbedding owns a shared embedding table, a relevance scorer, padding masks, and weighted pooling. ClickPredictor wraps it with the prediction head. Inputs are history IDs [B, N] and target IDs [B]; outputs are logits [B] and inspectable history weights [B, N]. This complete example includes initialization, one training update, and inference. B counts examples, N counts padded history positions, and D is the embedding width. A logit is an unrestricted score; sigmoid converts it to a click probability.
42 torch.manual_seed(0)43 # Two examples, each with one history and one candidate. ID 0 is padding.44 history_ids = torch.tensor([[1, 2, 0], [3, 4, 5]]) # [B=2, N=3]45 target_ids = torch.tensor([3, 2]) # [B=2]46 clicked = torch.tensor([1., 0.]) # Observed labels47 model = ClickPredictor(num_items=6, dim=8)48 optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)Candidate relevance
Ranking scores C candidates for each of B examples, so candidate_ids is [B, C]. The model itself doesn't change: flatten the candidate axis into the batch axis, score all B×C pairs in one call, then reshape the results back to [B, C]. Each candidate still gets its own history weights and interest vector. Each history has N positions, with ID 0 reserved for padding. The included model learns D-feature item vectors, scores each history item against its candidate, sums the weighted vectors, and maps that summary plus the candidate to a click logit. Sigmoid converts each logit to a probability. The file includes this entire model; no earlier example is required.
43 model = ClickPredictor(num_items=6, dim=8).eval()44 history_ids = torch.tensor([[1, 2, 0], [3, 4, 5]]) # [B=2, N=3]45 candidate_ids = torch.tensor([[3, 4], [1, 2]]) # [B=2, C=2]46 batch, candidates = candidate_ids.shape47 length = history_ids.shape[1]48 histories = history_ids[:, None, :].expand(batch, candidates, length)49 flat_histories = histories.reshape(batch * candidates, length)Summarize within categories before combining them
A single history may mix sportswear and kitchen products. First summarize relevant items within each known category; then decide which category summaries matter for the candidate. items [B, N, D] and target [B, D] contain feature vectors. category_ids [B, N] assigns one category 0…G−1 to each item; −1 marks padding. Output interest [B, D] is a feature vector, not a click probability. B is the batch size, N the padded history length, D the vector width, and G the number of categories. Category labels are supplied by the data; the network learns relevance, not the category assignments.
3 4class Relevance(nn.Module):5 def __init__(self, dim):6 super().__init__()7 self.net = nn.Sequential(nn.Linear(4 * dim, 16), nn.ReLU(), nn.Linear(16, 1))8 9 def forward(self, items, target):Train one model from IDs to click logits
Predict a click from item IDs and known categories. B is the number of examples, N the padded history length, D the learned embedding width, and G the category count. Supply history IDs [B, N], one target ID per example [B], and category IDs [B, N]. The model looks up D-feature vectors, averages relevant items within categories, then weights and sums those summaries for the candidate. A head returns logits [B]: unrestricted click scores. Observed binary labels [B] train the whole model with one loss.
34 35class HierarchicalRecommender(nn.Module):36 def __init__(self, num_items, dim, categories):37 super().__init__()38 self.embedding = nn.Embedding(num_items, dim, padding_idx=0)39 self.pool = HierarchicalInterest(dim, categories)40 self.head = nn.Linear(2 * dim, 1)Your place is saved automatically. Reaching the end marks this paper as read.
Page 1