Knowledge Graph Link Prediction Equations And Latex Code
rockingdingo 20221023 #Knowledge Graph #Link Prediction #KGKnowledge Graph Link Prediction Equations And Latex Code
Navigation
In this blog, we will summarize the latex code of most fundamental and popular knowledge graph (KG) Equations, with special focus on the link prediction tasks. We will cover a wide range of models, including TransE, TransR, TransE, RotatE, SME(Linear), SimplE etc. Knowledge Graph consists of a set of triples [(h, r, t)]. h and t denotes the head node and tail node respectively. And r denotes multiple relation types. One common solution to the link prediction tasks is to learn lowdimensional embeddings of entities(E) and relations (R), and infer the missing part of [(?, r, t), (h, ?, t), (h, r, ?)].
 1. Knowledge Graph Embeddings
 1.1 TransE
 1.2 TransH
 1.3 TransR
 1.4 RotatE
 1.5 SME(Linear)
 1.6 SimplE
1. Knowledge Graph Embeddings

1.1 TransE
Equation
Latex Code
\mathcal{L}=\sum_{(h,r,t) \in S} \sum_{(h^{'},r^{'},t^{'}) \in S^{'}_{(h,r,t)}} \[ \gamma + d(h + r, t)  d(h^{'} + r^{'}, t^{'}) \]_{+} \\ S^{'}_{(h,r,t)}=\{(h^{'},r,t)h^{'} \in E\} \cup \{(h,r,t^{'})t^{'} \in E\} \\ d(h + r, t)=h + r  t^{2}_{2}
Explanation
Given a training set S of triplets (h, l, t) composed of two entities h, t â?? E (the set of entities) and a relationship l â?? L (the set of relationships), our model learns vector embeddings of the entities and the relationships. The embeddings take values in Rk (k is a model hyperparameter) and are denoted with the same letters, in boldface characters. The basic idea behind our model is that the functional relation induced by the llabeled edges corresponds to a translation of the embeddings, i.e. we want that h + l â?? t when (h, l, t) holds (t should be a nearest neighbor of h + l), while h + l should be far away from t otherwise. Following an energybased framework, the energy of a triplet is equal to d(h + l, t) for some dissimilarity measure d, which we take to be either the L1 or the L2 norm. To learn such embeddings, we minimize a marginbased ranking criterion over the training set. See paper Translating Embeddings for Modeling Multirelational Data for more details.

1.2 TransH
Equation
Latex Code
f_{r}(h,t) =h_{\perp} + d_{r}  t_{\perp} ^{2}_{2}=(h  w_{r}hw_{r}) + d_{r}  (t  w_{r}tw_{r}) ^{2}_{2}
Explanation
TransH model learns lowdimensional representations of knowledge graphs triples on the hyperplane of the entities and relations. See paper Knowledge Graph Embedding by Translating on Hyperplanes for more details.

1.3 TransR
Equation
Latex Code
h_{r}=hM_{r}, t_{r}=tM_{r} \\f_{r}(h, t) = h_{r} + r  t_{r}^{2}_{2}=hM_{r}+rtM_{r}^{2}_{2}
Explanation
TransR model learns lowdimensional representations of entities and relations to relation space r, and multiple original entity embedding to the mapping matrix M. See paper Learning Entity and Relation Embeddings for Knowledge Graph Completion for more details.

1.4 RotatE
Equation
Latex Code
f_{r}(h, t) = h \circ r  t^{2}_{2}
Explanation
RotatE learns lowdimensional representations of entities and relations to relation space r, and multiple original entity embedding to the mapping matrix M. See paper RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space for more details.

1.5 SME(Linear)
Equation
Latex Code
\epsilon(lhs,rel,rhs)=E_{lhs(rel)}^{T}E_{rhs(rel)} \\=(W_{l1}E_{lhs}^{T} + W_{l2}E_{rel}^{T} + b_{l})^{T}(W_{r1}E_{rhs}^{T} + W_{r2}E_{rel}^{T} + b_{r})
Explanation
The energy function E (denoted SME) is encoded using a neural network, whose architecture first processes each entity in parallel, like in siamese networks. The intuition is that the relation type should first be used to extract relevant components from each argumentâ??s embedding, and put them in a space where they can then be compared. See paper A Semantic Matching Energy Function for Learning with Multirelational Data for more details.

1.6 SimplE
Equation
Latex Code
s(e_{i}, r, e_{j}) = \frac{1}{2}(â?¨h_{e_{i}}, v_{r}, t_{e_{j}}â?© + â?¨h_{e_{j}}, v_{r^{}}, t_{e_{i}}â?©)
Explanation
The similarity function for a triple (e1 , r , e2 ) is â?¨he1 , vr , te2 â?©. SimplE considers two vectors he,te â?? Rd as the embedding of each entity (similar to CP), and two vectors vr , vrâ??1 â?? Rd for each relation r. The similarity function of SimplE foratriple(ei,r,ej)isdefinedas 1(â?¨hei,vr,tejâ?©+â?¨hej,vrâ??1,teiâ?©),i.e. theaverageoftheCP 2 scoresfor(ei,r,ej)and(ej,râ??1,ei). See paper SimplE Embedding for Link Prediction in Knowledge Graphs for more details.