Loading your learning…
Loading your learning…
One training example contains an entire history plus one candidate. items [B, N, D] holds B examples; each example's history has N positions, and each position holds one item vector of D features. targets [B, D] holds one candidate per example. The scorer produces weights [B, N], pooling produces interest [B, D], and the prediction head produces B click logits. The main section runs all B examples together with tiny hand-written vectors and no padding.
import torchfrom torch import nn class LocalActivation(nn.Module): def __init__(self, dim): super().__init__() self.score = nn.Sequential( nn.Linear(4 * dim, 32), nn.PReLU(), nn.Linear(32, 1) ) def forward(self, items, targets): # items: [B, N, D]; targets: [B, D]. query = targets[:, None, :].expand_as(items) features = torch.cat([items, query, items - query, items * query], dim=-1) return self.score(features).squeeze(-1) # [B, N] if __name__ == "__main__": torch.manual_seed(0) items = torch.tensor([[[1., 0.], [0., 2.]], [[0., 1.], [2., 0.]]]) # [B=2, N=2, D=2] targets = torch.tensor([[1., 0.5], [0.5, 1.]]) # [B, D] activation = LocalActivation(dim=2) weights = activation(items, targets) # [B, N] interest = (weights[..., None] * items).sum(dim=1) # [B, D] predictor = nn.Linear(4, 1) logits = predictor(torch.cat([interest, targets], dim=-1)).squeeze(-1) print("History weights:", weights.detach()) print("Interest vectors:", interest.detach()) print("Candidate probabilities:", logits.sigmoid().detach()) # These randomly initialized networks have not learned relevance yet.The example uses B=2 examples, N=2 history positions per example, and D=2 features per item. LocalActivation(dim=2) creates one shared scorer; activation(items, targets) runs it on all four history-item/candidate pairs. Initialization is random, so these scores do not yet express learned relevance.
return self.score(features).squeeze(-1) # [B, N] if __name__ == "__main__": torch.manual_seed(0) items = torch.tensor([[[1., 0.], [0., 2.]], [[0., 1.], [2., 0.]]]) # [B=2, N=2, D=2] targets = torch.tensor([[1., 0.5], [0.5, 1.]]) # [B, D] activation = LocalActivation(dim=2) weights = activation(items, targets) # [B, N] interest = (weights[..., None] * items).sum(dim=1) # [B, D] predictor = nn.Linear(4, 1)Each history item needs to be compared with its example’s target. items has shape [B, N, D], but targets has only [B, D]: one D-feature target per example, not one per history position. targets[:, None, :] inserts a size-one axis in the history-position slot, giving [B, 1, D]. expand_as(items) then expands that axis from 1 to N so query has the same shape as items: [B, N, D]. It uses items only to determine the shape; it does not copy item values into query. For B = 2, N = 3, D = 2 and targets = [[1, 2], [7, 8]], query looks like: [[[1, 2], [1, 2], [1, 2]], [[7, 8], [7, 8], [7, 8]]] Each example keeps its own target, repeated logically across its three history positions. We can now subtract and multiply item and target coordinates, and concatenate their features. Expansion creates a view that reuses the target’s storage rather than allocating N copies. A size-one axis can expand this way; an arbitrary axis of another size cannot simply be stretched to N.
) def forward(self, items, targets): # items: [B, N, D]; targets: [B, D]. query = targets[:, None, :].expand_as(items) features = torch.cat([items, query, items - query, items * query], dim=-1) return self.score(features).squeeze(-1) # [B, N]