Paper → implementation
See this paper in code.
Community-written snippets that make the paper’s ideas concrete. These are not official implementations from the paper’s authors.
Multiply features to express interactions
Suppose a prediction depends on the combination of two features, not just their separate effects. A cross layer builds those products explicitly. Input x [B, D] has B examples and D numerical features per example; output [B, D] contains transformed features, not probabilities. Here B = 2 and D = 2. Real inputs could concatenate normalized numerical features and learned category embeddings.
21 22if __name__ == "__main__":23 torch.manual_seed(0)24 x = torch.tensor([[2., 3.], [1., 4.]]) # [B=2, D=2]25 cross = CrossNetworkV1(dim=2, depth=1)26 with torch.no_grad():27 cross.weights[0].copy_(torch.tensor([0.5, 1.]))Use cross features in a complete predictor
Predict a click using interactions between person, item, and context features. Input x [B, D] contains B examples with D numeric features each. A cross branch repeatedly scales the original features by a learned weighted sum of the current features, then adds a bias and the previous result. A separate deep branch transforms the same input with a linear layer and ReLU. Concatenate both outputs and use a head to return B logits: unrestricted click scores. The main section trains the complete model for one illustrative update.
21 22class DeepCrossPredictor(nn.Module):23 def __init__(self, dim):24 super().__init__()25 self.cross = CrossNetworkV1(dim, depth=2)26 self.deep = nn.Sequential(nn.Linear(dim, 16), nn.ReLU())27 self.head = nn.Linear(dim + 16, 1)Your place is saved automatically. Reaching the end marks this paper as read.
Page 1