Knowledge Graph Link Prediction Equations And Latex Code

Navigation

In this blog, we will summarize the latex code of most fundamental and popular knowledge graph (KG) Equations, with special focus on the link prediction tasks. We will cover a wide range of models, including TransE, TransR, TransE, RotatE, SME(Linear), SimplE etc. Knowledge Graph consists of a set of triples [(h, r, t)]. h and t denotes the head node and tail node respectively. And r denotes multiple relation types. One common solution to the link prediction tasks is to learn low-dimensional embeddings of entities(E) and relations (R), and infer the missing part of [(?, r, t), (h, ?, t), (h, r, ?)].

• 1. Knowledge Graph Embeddings
• 1.1 TransE
• 1.2 TransH
• 1.3 TransR
• 1.4 RotatE
• 1.5 SME(Linear)
• 1.6 SimplE

• 1.1 TransE

  Equation

  
  
  

  Latex Code

              \mathcal{L}=\sum_{(h,r,t) \in S} \sum_{(h^{'},r^{'},t^{'}) \in S^{'}_{(h,r,t)}} \[ \gamma + d(h + r, t) - d(h^{'} + r^{'}, t^{'}) \]_{+} \\ S^{'}_{(h,r,t)}=\{(h^{'},r,t)|h^{'} \in E\} \cup \{(h,r,t^{'})|t^{'} \in E\} \\ d(h + r, t)=||h + r - t||^{2}_{2}
          

  Explanation

  Given a training set S of triplets (h, l, t) composed of two entities h, t â?? E (the set of entities) and a relationship l â?? L (the set of relationships), our model learns vector embeddings of the entities and the relationships. The embeddings take values in Rk (k is a model hyperparameter) and are denoted with the same letters, in boldface characters. The basic idea behind our model is that the functional relation induced by the l-labeled edges corresponds to a translation of the embeddings, i.e. we want that h + l â?? t when (h, l, t) holds (t should be a nearest neighbor of h + l), while h + l should be far away from t otherwise. Following an energy-based framework, the energy of a triplet is equal to d(h + l, t) for some dissimilarity measure d, which we take to be either the L1 or the L2 -norm. To learn such embeddings, we minimize a margin-based ranking criterion over the training set. See paper Translating Embeddings for Modeling Multi-relational Data for more details.

• 1.2 TransH

  Equation

  
  
  

  Latex Code

              f_{r}(h,t) =||h_{\perp} + d_{r} - t_{\perp} ||^{2}_{2}=||(h - w_{r}hw_{r}) + d_{r} - (t - w_{r}tw_{r}) ||^{2}_{2}
          

  Explanation

  TransH model learns low-dimensional representations of knowledge graphs triples on the hyperplane of the entities and relations. See paper Knowledge Graph Embedding by Translating on Hyperplanes for more details.

• 1.3 TransR

  Equation

  
  
  

  Latex Code

              h_{r}=hM_{r}, t_{r}=tM_{r} \\f_{r}(h, t) = ||h_{r} + r - t_{r}||^{2}_{2}=||hM_{r}+r-tM_{r}||^{2}_{2}
          

  Explanation

  TransR model learns low-dimensional representations of entities and relations to relation space r, and multiple original entity embedding to the mapping matrix M. See paper Learning Entity and Relation Embeddings for Knowledge Graph Completion for more details.

• 1.4 RotatE

  Equation

  
  
  

  Latex Code

              f_{r}(h, t) = ||h \circ r - t||^{2}_{2}
          

  Explanation

  RotatE learns low-dimensional representations of entities and relations to relation space r, and multiple original entity embedding to the mapping matrix M. See paper RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space for more details.

• 1.5 SME(Linear)

  Equation

  
  
  

  Latex Code

              \epsilon(lhs,rel,rhs)=E_{lhs(rel)}^{T}E_{rhs(rel)} \\=(W_{l1}E_{lhs}^{T} + W_{l2}E_{rel}^{T} + b_{l})^{T}(W_{r1}E_{rhs}^{T} + W_{r2}E_{rel}^{T} + b_{r})
          

  Explanation

  The energy function E (denoted SME) is encoded using a neural network, whose architecture first processes each entity in parallel, like in siamese networks. The intuition is that the relation type should first be used to extract relevant components from each argumentâ??s embedding, and put them in a space where they can then be compared. See paper A Semantic Matching Energy Function for Learning with Multi-relational Data for more details.

• 1.6 SimplE

  Equation

  
  
  

  Latex Code

              s(e_{i}, r, e_{j}) = \frac{1}{2}(â?¨h_{e_{i}}, v_{r}, t_{e_{j}}â?© + â?¨h_{e_{j}}, v_{r^{-}}, t_{e_{i}}â?©)
          

  Explanation

  The similarity function for a triple (e1 , r , e2 ) is â?¨he1 , vr , te2 â?©. SimplE considers two vectors he,te â?? Rd as the embedding of each entity (similar to CP), and two vectors vr , vrâ??1 â?? Rd for each relation r. The similarity function of SimplE foratriple(ei,r,ej)isdefinedas 1(â?¨hei,vr,tejâ?©+â?¨hej,vrâ??1,teiâ?©),i.e. theaverageoftheCP 2 scoresfor(ei,r,ej)and(ej,râ??1,ei). See paper SimplE Embedding for Link Prediction in Knowledge Graphs for more details.