DeepNLP Equation: Equation Database

Navigation

economics
nlp
physics
ai
math
machine learning
statistics

Equation Database

economics

Black-Derman-Toy BDT
Gross Domestic Product Deflator
Marginal Revenue Product of Labor MRPL
Merchandise Trade Balance
Nominal Interest Rate
Optimal Combination of Resources Condition
Real GDP
Real Interest Rate

BLEU Bilingual Evaluation Understudy
Binary Cross Entropy Optimization BCO
Conditional Random Field CRF
Contrastive Preference Optimization CPO
Denoising Diffusion Policy Optimization DDPO
Direct Policy Optimization DPO
Direct Preference Optimization DPO
Generalized Knowledge Distillation GKD
KTO Kahneman-Tversky Optimisation Equation
LOW RANK ADAPTATION LORA

Harmonic Oscillator Quantum
The Uncertainty Principle
Youngs Modulus

Binary Cross Entropy Optimization BCO
Contrastive Preference Optimization CPO
Denoising Diffusion Policy Optimization DDPO
Generalized Knowledge Distillation GKD
KTO Kahneman-Tversky Optimisation Equation
LOW RANK ADAPTATION LORA
Odds Ratio Preference Optimization ORPO
Proximal Policy Optimization PPO
RLHF Reinforcement Learning from Human Feedback

Arithmetic and Geometric Progressions
Bessel Equation
Binomial Expansion
Complex Numbers
Convergence of Series
De Moivre's Theorem
Diffusion Conduction Equation
Gamma Distribution
Heat Equation
Laplace Equation

Bellman Equation
Bound on Target Domain Error
Conditional Random Field CRF
Cross-Stitch Network
Deep Kernel Learning
Domain Adaptation H-Divergence
Domain-Adversarial Neural Networks DANN
Entire Space Multi-Task Model ESSM
Generative Adversarial Networks GAN
Graph Attention Network GAT

Gamma Distribution
Poisson Binomial Distribution
Student t-Distribution

economics

Black-Derman-Toy BDT

#Financial #Economics

$$\text{First Node: 1-year bond price} \\ P_{0} = \frac{1}{1 + R_{0}} \\ \text{Second Node} \\ P_{1} = \frac{1}{1+R_{0}} [\frac{1}{2} P(1,2,r_{u}) + \frac{1}{2} P(1,2,r_{d})] \\ = \frac{1}{1+R_{0}} [\frac{1}{2(1 + R_{1}e^{2\sigma_{1}})} + \frac{1}{2(1 + R_{1})}] \\ R_{0} = \frac{1}{2} \ln (\frac{R_{1} e^{2\sigma_{1}} }{R_{1}})$$

Gross Domestic Product Deflator

#Economics #MacroEconomics

$$\text{GDP Deflator}= \frac{\text{Current Year Quantities} \times \text{Current Year Prices}}{\text{Current Year Quantities} \times \text{Base Year Prices}} \times 100$$

Marginal Revenue Product of Labor MRPL

#Economics #Microeconomics

$$\text{MRP}_{L} = \text{MP}_{L} \times \text{P}$$

Merchandise Trade Balance

#Economics #MacroEconomics

$$\text{Merchandise Trade Balance}=\text{Value of Merchandise Exports} - \text{Value of Merchandise Imports}$$

Nominal Interest Rate

#Economics #MacroEconomics

$$\text{Nominal Interest Rate}=\text{Real Interest Rate} + \text{Anticipated Inflation}$$

Optimal Combination of Resources Condition

#Economics #Microeconomics

$$\text{MRP}_{L} = \text{MP}_{L} \times \text{P}$$

Real GDP

#Economics #MacroEconomics

$$\text{Real GDP}=\frac{Nominal GDP}{CPI for the same year as the nominal figure} \times 100$$

Real Interest Rate

#Economics #MacroEconomics

$$\text{Real Interest Rate} = \text{Nominal Interest Rate} – \text{Anticipated Inflation}$$

BLEU Bilingual Evaluation Understudy

#nlp #BLEU #evaluation

$$ \text{BLEU}_{w}(\hat{S},S)=BP(\hat{S};S) \times \exp{\sum^{\infty}_{n=1}w_{n} \ln p_{n}(\hat{S};S)}, p_{n}(\hat{S};S)=\frac{\sum^{M}_{i=1}\sum_{s \in G_{n}(\hat{y})} \min(C(s,\hat{y}),\max_{y \in S_{i}} C(s,y))}{\sum^{M}_{i=1}\sum_{s \in G_{n}(\hat{y})}C(s, \hat{y})}, p_{n}(\hat{y};y)=\frac{\sum_{s \in G_{n}(\hat{y})} \min(C(s,\hat{y}), C(s,y))}{\sum_{s \in G_{n}(\hat{y})}C(s, \hat{y})}, BP(\hat{S};S) = e^{-(r/c-1)^{+}}$$

Binary Cross Entropy Optimization BCO

#AI #nlp #llm #RLHF

$$E_{(x, y_w, y_l) \sim \mathcal{D}} [-\log \sigma \left( r_\theta (x, y_w) - r_\theta(x, y_l) \right) ] < E_{(x, y_w, y_l) \sim \mathcal{D}} [- \log \sigma (r_\theta(x, y_w))] + E_{(x, y_w, y_l) \sim \mathcal{D}} [- \log \left( 1 - \sigma (r_\theta (x, y_l)) \right)] $$ $$ E_{(x, y_w, y_l) \sim \mathcal{D}} [- \log \sigma(r_\theta(x, y_w) - \delta) - \log \sigma(- (r_\theta(x, y_l) - \delta))] $$ $$ \mathcal{L}_\text{BCO}(\theta) = - E_{(x, y) \sim \mathcal{D}^+} [\log \sigma (r_\theta (x, y) - \delta)] - E_{(x, y) \sim \mathcal{D}^-} \left[ \frac{p_\psi (f = 1 \mid x)}{p_\psi (f = 0 \mid x)} \log \sigma (- (r_\theta (x, y) - \delta)) \right] $$

Conditional Random Field CRF

#machine learning #nlp

$$P(y|x)=\frac{1}{Z(x)}\exp(\sum_{i,k}\lambda_{k}t_{k}(y_{i-1},y_{i},x,i))+\sum_{i,l}\mu_{l}s_{l}(y_{i},x,i)) \\ Z(x)=\sum_{y}\exp(\sum_{i,k}\lambda_{k}t_{k}(y_{i-1},y_{i},x,i))+\sum_{i,l}\mu_{l}s_{l}(y_{i},x,i))$$

Contrastive Preference Optimization CPO

#AI #nlp #llm #RLHF

$$\mathcal{L}(\pi_\theta;\pi_{\text{ref}}) = -\mathbb{E}_{(x,y_w,y_l) \sim \mathcal{D}} \Big[ \log \sigma \Big( \beta \log \frac{\pi_{\theta}(y_w | x)}{\pi_{\text{ref}}(y_w | x)} - \beta \log \frac{\pi_{\theta}(y_l | x)}{\pi_{\text{ref}} (y_l | x)} \Big) \Big] $$ $$ \mathcal{L}(\pi_\theta;U) = -\mathbb{E}_{(x,y_w,y_l) \sim \mathcal{D}} \Big[ \log \sigma \Big( \beta \log \pi_{\theta}(y_w | x) \nonumber \\ - \beta \log \pi_{\theta}(y_l | x) \Big) \Big] $$ $$ \min_\theta \mathcal{L}(\pi_\theta, U) \notag \text{ s.t. } \mathbb{E}_{(x,y_w) \sim \mathcal{D}}\Big [ \mathbb{KL}(\pi_w(y_w|x)||\pi_\theta(y_w|x))\Big] < \epsilon $$ $$ \min_\theta\underbrace{ \mathcal{L}(\pi_\theta, U)}_{\mathcal{L}_\text{prefer}} \underbrace{-\mathbb{E}_{(x,y_w) \sim \mathcal{D}} [\log \pi_\theta(y_w| x)]}_{\mathcal{L}_\text{NLL}} $$

Denoising Diffusion Policy Optimization DDPO

#AI #nlp #llm #RLHF

$$\mathcal{J}_\text{DDRL}(\theta) = \mathbb{E}_{c \sim p(c), x_{0} \sim p_{\theta} (x_{0} | c)} [r(x_{0}, c)] $$ $$ w_{\text{RWR}}(x_0, c) = \frac{1}{Z} \exp\big(\beta r(x_0, c) \big) $$ $$ w_{\text{sparse}} (x_0, c) = \mathbf{1} \big[ r(x_0, c) \geq C \big] $$ $$ \nabla_\theta \mathcal{J}_\text{DDRL} = \mathbb{E} {\; \sum_{t=0}^{T} \nabla_\theta \log p_\theta(x_{t-1} \mid x_t, c) \; r(x_0, c)} $$ $$ \nabla_\theta \mathcal{J}_\text{DDRL} = \mathbb{E} {\; \sum_{t=0}^{T} \frac{p_\theta (x_{t-1} \mid x_t, c)}{p_{\theta_\text{old}} (x_{t-1} \mid x_t, c)} \; \nabla_\theta \log p_\theta(x_{t-1} \mid x_t, c) \; r(x_0, c)} $$

Direct Policy Optimization DPO

#nlp #llm #RLHF

$$\pi_{r} (y|x) = \frac{1}{Z(x)} \pi_{ref} (y|x) \exp(\frac{1}{\beta} r(x,y) ) , r(x,y) = \beta \log \frac{\pi_{r} (y|x)}{\pi_{ref} (y|x)} + \beta \log Z(x) , p^{*}(y_{1} > y_{2} |x) = \frac{1}{1+\exp{(\beta \frac{\pi^{*} (y_{2}|x)}{\pi_{ref} (y_{2}|x)} - \beta \frac{\pi^{*} (y_{1}|x)}{\pi_{ref} (y_{1}|x)} )}} , \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = -\mathbb{E}_{(x, y_{w},y_{l}) \sim D } [\log \sigma (\beta \frac{\pi_{\theta} (y_{w}|x)}{\pi_{ref} (y_{w}|x)} - \beta \frac{\pi_{\theta} (y_{l}|x)}{\pi_{ref} (y_{l}|x)} )] , \nabla \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = - \beta \mathbb{E}_{(x, y_{w},y_{l}) \sim D } [ \sigma ( \hat{r}_{\theta} (x, y_{l}) - \hat{r}_{\theta} (x, y_{w})) [\nabla_{\theta} \log \pi (y_{w}|x) - \nabla_{\theta} \log \pi (y_{l}|x) ] ] , \hat{r}_{\theta} (x, y) = \beta \log (\frac{\pi_{\theta} (y|x)}{\pi_{ref} (y|x)})$$

Direct Preference Optimization DPO

#nlp #llm #RLHF

$$\pi_{r} (y|x) = \frac{1}{Z(x)} \pi_{ref} (y|x) \exp(\frac{1}{\beta} r(x,y) ) $$ $$ r(x,y) = \beta \log \frac{\pi_{r} (y|x)}{\pi_{ref} (y|x)} + \beta \log Z(x) $$ $$ p^{*}(y_{1} > y_{2} |x) = \frac{1}{1+\exp{(\beta \frac{\pi^{*} (y_{2}|x)}{\pi_{ref} (y_{2}|x)} - \beta \frac{\pi^{*} (y_{1}|x)}{\pi_{ref} (y_{1}|x)} )}} $$ $$ \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = -\mathbb{E}_{(x, y_{w},y_{l}) \sim D } [\log \sigma (\beta \frac{\pi_{\theta} (y_{w}|x)}{\pi_{ref} (y_{w}|x)} - \beta \frac{\pi_{\theta} (y_{l}|x)}{\pi_{ref} (y_{l}|x)} )] $$ $$ \nabla \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = - \beta \mathbb{E}_{(x, y_{w},y_{l}) \sim D } [ \sigma ( \hat{r}_{\theta} (x, y_{l}) - \hat{r}_{\theta} (x, y_{w})) [\nabla_{\theta} \log \pi (y_{w}|x) - \nabla_{\theta} \log \pi (y_{l}|x) ] ] $$ $$ \hat{r}_{\theta} (x, y) = \beta \log (\frac{\pi_{\theta} (y|x)}{\pi_{ref} (y|x)})$$

Generalized Knowledge Distillation GKD

#AI #nlp #llm #RLHF

$$L_\mathrm{GKD}(\theta) := (1 - \lambda) \mathbb{E}_{(x, y) \sim (X, Y)} \big[ \mathcal{D}(p_{T} \| p_{S}^\theta)(y|x) \big] + \lambda \mathbb{E}_{x\sim X} \Big[\mathbb{E}_{y \sim p_{S} (\cdot|x)} \big[\mathcal{D}(p_{T} \| p_{S}^\theta)(y|x)\big]\Big] $$

KTO Kahneman-Tversky Optimisation Equation

#nlp #llm #AI

$$f(\pi_\theta, \pi_\text{ref}) = \mathbb{E}_{x,y\sim\mathcal{D}}[ a_{x,y} v(r_\theta(x,y) - \mathbb{E}_{Q}[r_\theta(x, y')])] + C_\mathcal{D}$$

LOW RANK ADAPTATION LORA

#AI #nlp #llm #RLHF

$$W_{0} + \Delta W_{0} = W_{0} + BA, h=W_{0}x + \Delta W_{0}x = W_{0}x + BAx, \text{Initialization:} A \sim N(0, \sigma^{2}), B = 0$$

Odds Ratio Preference Optimization ORPO

#AI #nlp #llm #RLHF

$$\mathcal{L}_{ORPO} = \mathbb{E}_{(x, y_w, y_l)}\left[ \mathcal{L}_{SFT} + \lambda \cdot \mathcal{L}_{OR} \right] $$ $$ \mathcal{L}_{OR} = -\log \sigma \left( \log \frac{\textbf{odds}_\theta(y_w|x)}{\textbf{odds}_\theta(y_l|x)} \right) $$

RLHF Reinforcement Learning from Human Feedback

#AI #nlp #LLM #equation

$$p^*(y_w \succ y_l|x) = \sigma(r^*(x,y_w) - r^*(x,y_l)) $$ $$ \mathcal{L}_R(r_\phi) = \mathbb{E}_{x,y_w,y_l \sim D}[- \log \sigma(r_\phi(x, y_w) - r_\phi(x, y_l))] $$ $$ \mathbb{E}_{x \in D, y \in \pi_\theta} [r_\phi(x,y)] - \beta D_{\text{KL}}(\pi_\theta(y|x) \| \pi_{\text{ref}}(y|x)) $$

Harmonic Oscillator Quantum

#physics #quantum

$$H=\frac{p^2}{2m}+\frac{1}{2} m\omega^2 x^2= \frac{1}{2} \hbar\omega+\omega A^\dagger A \\ A=\sqrt{\mbox{$\frac{1}{2}$}m\omega}x+\frac{ip}{\sqrt{2m\omega}} \\ A^\dagger=\sqrt{\mbox{$\frac{1}{2}$}m\omega}x-\frac{ip}{\sqrt{2m\omega}} \\ HAu_E=(E-\hbar\omega)Au_E \\ u_n=\frac{1}{\sqrt{n!}}\left(\frac{A^\dagger}{\sqrt{\hbar}}\right)^nu_0 \\ u_0=\sqrt[4]{\frac{m\omega}{\pi\hbar}}\exp\left(-\frac{m\omega x^2}{2\hbar}\right) \\ E_n=( \frac{1}{2} +n)\hbar\omega$$

The Uncertainty Principle

#physics #quantum

$$(\Delta A)^2=\left\langle \psi|A_{\rm op}-\left\langle A \right\rangle|^2\psi \right\rangle=\left\langle A^2 \right\rangle-\left\langle A \right\rangle^2 \\ \Delta A\cdot\Delta B\geq \frac{1}{2} |\left\langle \psi|[A,B]|\psi \right\rangle| \\ \Delta E\cdot\Delta t\geq\hbar \\ \Delta p_x\cdot\Delta x\geq \frac{1}{2} \hbar \\ \Delta L_x\cdot\Delta L_y\geq \frac{1}{2} \hbar L_z$$

Youngs Modulus

#physics #mechanics #youngs modulus

$$\Upsilon = \frac{F/A}{\Delta L/L} =\frac{\text{Stress}}{\text{Strain}}$$

Binary Cross Entropy Optimization BCO

#AI #nlp #llm #RLHF

Contrastive Preference Optimization CPO

#AI #nlp #llm #RLHF

Denoising Diffusion Policy Optimization DDPO

#AI #nlp #llm #RLHF

Generalized Knowledge Distillation GKD

#AI #nlp #llm #RLHF

KTO Kahneman-Tversky Optimisation Equation

#nlp #llm #AI

$$f(\pi_\theta, \pi_\text{ref}) = \mathbb{E}_{x,y\sim\mathcal{D}}[ a_{x,y} v(r_\theta(x,y) - \mathbb{E}_{Q}[r_\theta(x, y')])] + C_\mathcal{D}$$

LOW RANK ADAPTATION LORA

#AI #nlp #llm #RLHF

$$W_{0} + \Delta W_{0} = W_{0} + BA, h=W_{0}x + \Delta W_{0}x = W_{0}x + BAx, \text{Initialization:} A \sim N(0, \sigma^{2}), B = 0$$

Odds Ratio Preference Optimization ORPO

#AI #nlp #llm #RLHF

Proximal Policy Optimization PPO

#machine learning #AI #LLM

$$\arg\max\limits_{\pi}{ E_{p \sim D,g \sim \pi} [R(g|p)] }, R(g|p) = \tilde{R}_{c}(g|p) - \beta D_{KL}( \pi_{\theta} (g|p) || \pi_{0} (g|p))$$

RLHF Reinforcement Learning from Human Feedback

#AI #nlp #LLM #equation

Arithmetic and Geometric Progressions

#math #arithmetic #geometric #progressions

$$S_{n}=a+(a+d)+(a+2d)+...+[a+(n-1)d]=\frac{n}{2}[2a+(n-1)d] \\ S_{n}=a+ar+ar^{2}+...+ar^{n-1}=a\frac{1-r^{n}}{1-r}$$

Bessel Equation

#math #bessel equation

$$x^{2}\frac{\mathrm{d}^{2} y}{\mathrm{d} x^{2}}+x\frac{\mathrm{d} y}{\mathrm{d} x}+(x^{2}-m^{2})y=0 \\ J_{m}(x)=\sum^{\infty}_{k=0}\frac{(-1)^{k} (x/2)^{m+2k}}{k!(m+k)!}$$

Binomial Expansion

#math #binomial #expansion

$$(1+x)^{n}=1+nx+\frac{n(n-1)}{2!}x^{2}+...+ C^{r}_{n}x^{r} +...+x^{n} \\ C^{r}_{n}=\frac{n!}{r!(n-r)!}$$

Complex Numbers

#math #complex numbers

$$z=x+iy=r(\cos \theta + i \sin \theta)=re^{i(\theta+2n\pi)} \\ z^{*}=x-iy=r(\cos \theta - i \sin \theta)=re^{-i\theta} \\ zz^{*}=|z|^{2}=x^{2}+y^{2}$$

Convergence of Series

#math #convergence #series

$$S_{n}=u_{1}+u_{2}+...+u_{n} \\ \text{Converge AS } n \rightarrow \infty \text{, If} \lim_{n \rightarrow \infty} |\frac{u_{n+1}}{u_{n}}| < 1$$

De Moivre's Theorem

#math #complex number #De Moivre

$$(\cos \theta + i \sin \theta)^{n}=e^{in\theta}=\cos n\theta + i \sin n\theta$$

Diffusion Conduction Equation

#math #diffusion #conduction

$$\kappa u_{xx} = u_{t} \\ \frac{\partial \Psi}{\partial t}=\kappa \triangledown^{2} \Psi$$

Gamma Distribution

#Math #Statistics

$$\Gamma \left( a \right) = \int\limits_0^\infty {s^{a - 1} } e^{ - s} ds \\ P(x) = \frac{x^{\alpha-1} e^{-frac{x}{\theta}}}{\Gamma(\alpha) \theta^{\alpha}} \\ \mu = \alpha \theta \\ \sigma^{2} = \alpha \theta^{2} \\ \gamma_{1} = \frac{2}{\sqrt{\alpha}} \\ \gamma_{2} = \frac{6}{\alpha}$$

Heat Equation

#math #heat equation

$$u_{t}={\alpha}^{2}u_{xx}$$

Laplace Equation

#math #laplace equation

$$u_{xx} = 0$$

Legendre Equation

#math #legendre equation

$$(1-x^{2})\frac{\mathrm{d}^{2} y}{\mathrm{d} x^{2}}-2x\frac{\mathrm{d} y}{\mathrm{d} x}+l(l+1)y=0 \\ P_{l}(x)=\frac{1}{2^{l}l!}(\frac{\mathrm{d}}{\mathrm{d} x})^{l}(x^2-1)^{l}\\ P_{l}(x)=\frac{1}{l}[(2l-1)xP_{l-1}(x)-(l-1)P_{l-2}(x)]$$

Plane Wave Expansion

#math #plane wave expansion

$$\exp(ikz)=\exp(ikr\cos \theta=\sum^{\infty}_{l=0}(2l+1)i^{l}j_{l}(kr)P_{l}(\cos \theta)$$

Poisson Binomial Distribution

#Math #Statistics

$$Pr(K = k) = \sum_{A \in F_{k}} \prod_{i \in A} p_{i} \prod_{j \in A_{c}} (1-p_{j})$$

Power Series with Real Variables

#math #power #series

$$e^{x}=1+x+\frac{x^{2}}{2!}+...+\frac{x^{n}}{n!}+... \\ \ln(1+x) = x - \frac{x^{2}}{2} + \frac{x^{3}}{3} + ... + (-1)^{n+1}\frac{x^{n}}{n!} +... \\ \cos(x) = \frac{e^{ix}+e^{-ix}}{2}=1-\frac{x^{2}}{2!}+\frac{x^{4}}{4!}-\frac{x^{6}}{6!}+...\\ \sin(x) = \frac{e^{ix}-e^{-ix}}{2i}=x-\frac{x^{3}}{3!}+\frac{x^{5}}{5!}+...$$

Regression Least Squares Fitting

#math #regression

$$y_{i}=\alpha + \beta (x_{i} - \bar{x}) + \epsilon_{i} \\ \hat{\alpha}=\bar{y} \\ \hat{\beta}=\frac{s^{2}_{xy}}{s^{2}_{x}}$$

Riemann Hypothesis

#math

$$\zeta (s)=\sum _{n=1}^{\infty }{\frac {1}{n^{s}}}={\frac {1}{1^{s}}}+{\frac {1}{2^{s}}}+{\frac {1}{3^{s}}}+\cdots $$ $$ \zeta (s)=\prod _{p{\text{ prime}}}{\frac {1}{1-p^{-s}}}={\frac {1}{1-2^{-s}}}\cdot {\frac {1}{1-3^{-s}}}\cdot {\frac {1}{1-5^{-s}}}\cdot {\frac {1}{1-7^{-s}}}\cdots$$

Spherical Harmonics Equation

#math #spherical harmonics

$$[\frac{1}{\sin \theta} \frac{\partial}{\partial \theta}(\sin \theta \frac{\partial}{\partial \theta}) + \frac{1}{\sin^{2} \theta} \frac{\partial^{2}}{\partial \phi^{2}}) ] Y^{m}_{l} + l(l+1) Y^{m}_{l}=0 \\ Y^{m}_{l}(\theta,\phi)=\sqrt{\frac{2l+1}{4 \pi} \frac{(l-|m|)!}{(l+|m|)!}}P^{m}_{l}(\cos \theta) e^{im \phi} \times \begin{cases}(-1)^{m} & m\ge 0 \\ 1 & m <0 \end{cases}$$

Student t-Distribution

#Math #Statistics

$$f(t) = \frac{\Gamma(\frac{v + 1}{2})}{\sqrt{v\pi}\Gamma(v/2)} (1+\frac{t^{2}}{v})^{-(v+1)/2}, \\ F(t) = \frac{1}{2} + x \Gamma(\frac{v+1}{2}) \times \frac{2^{F_{1}}(1/2,\frac{v+1}{2};3/2;-\frac{x^{2}}{v})}{\sqrt{\pi v} \Gamma(\frac{v}{2})}, \\ F(t) = 1 - \frac{1}{2} I_{x(t)} (\frac{v}{2}, \frac{1}{2}), x(t)=\frac{v}{t^{2} + v}$$

Wave Equation

#math #wave equation

$$u_{tt}=c^{2}u_{xx}$$

gaussian process

#math #gaussian process

$$\log p(y|X) \propto -[y^{T}(K + \sigma^{2}I)^{-1}y+\log|K + \sigma^{2}I|] \\ f(X)=\[f(x_{1}),f(x_{2}),...,f(x_{N}))\]^{T} \sim \mathcal{N}(\mu, K_{X,X}) \\ f_{*}|X_{*},X,y \sim \mathcal{N}(\mathbb{E}(f_{*}),\text{cov}(f_{*})) \\ \text{cov}(f_{*})=K_{X_{*},X_{*}}-K_{X_{*},X}[K_{X,X}+\sigma^{2}I]^{-1}K_{X,X_{*}}$$

Bellman Equation

#machine learning

$$v_{\pi}(s)=\sum_{a}\pi(a|s)\sum_{s^{'},r}p(s^{'},r|s,a)[r+\gamma v_{\pi}(s^{'})]$$

Bound on Target Domain Error

#machine learning #transfer learning

$$\epsilon_{T}(h) \le \hat{\epsilon}_{S}(h) + \sqrt{\frac{4}{m}(d \log \frac{2em}{d} + \log \frac{4}{\delta })} + d_{\mathcal{H}}(\tilde{\mathcal{D}}_{S}, \tilde{\mathcal{D}}_{T}) + \lambda \\ \lambda = \lambda_{S} + \lambda_{T}$$

Conditional Random Field CRF

#machine learning #nlp

Cross-Stitch Network

#machine learning #multi task

$$\begin{bmatrix} \tilde{x}^{ij}_{A}\\\tilde{x}^{ij}_{B}\end{bmatrix}=\begin{bmatrix} a_{AA} & a_{AB}\\ a_{BA} & a_{BB} \end{bmatrix}\begin{bmatrix} x^{ij}_{A}\\ x^{ij}_{B} \end{bmatrix}$$

Deep Kernel Learning

#machine learning #Deep Kernel Learning

$$k(x_{i},x_{j}|\phi)=k(h(x_i,w_k),h(x_j,w_k)|w_k,\phi)$$

Domain Adaptation H-Divergence

#machine learning #transfer learning

$$d_{\mathcal{H}}(\mathcal{D},\mathcal{D}^{'})=2\sup_{h \in \mathcal{H}}|\Pr_{\mathcal{D}}[I(h)]-\Pr_{\mathcal{D}^{'}}[I(h)]|$$

Domain-Adversarial Neural Networks DANN

#machine learning #transfer learning

$$\min [\frac{1}{m}\sum^{m}_{1}\mathcal{L}(f(\textbf{x}^{s}_{i}),y_{i})+\lambda \max(-\frac{1}{m}\sum^{m}_{i=1}\mathcal{L}^{d}(o(\textbf{x}^{s}_{i}),1)-\frac{1}{m^{'}}\sum^{m^{'}}_{i=1}\mathcal{L}^{d}(o(\textbf{x}^{t}_{i}),0))]$$

Entire Space Multi-Task Model ESSM

#machine learning #multi task

$$L(\theta_{cvr},\theta_{ctr})=\sum^{N}_{i=1}l(y_{i},f(x_{i};\theta_{ctr}))+\sum^{N}_{i=1}l(y_{i}\&z_{i},f(x_{i};\theta_{ctr}) \times f(x_{i};\theta_{cvr})) $$

Generative Adversarial Networks GAN

#machine learning #gan

$$\min_{G} \max_{D} V(D,G)=\mathbb{E}_{x \sim p_{data}(x)}[\log D(x)]+\mathbb{E}_{z \sim p_{z}(z)}[\log(1-D(G(z)))]$$

Graph Attention Network GAT

#machine learning #graph #GNN

$$h=\{\vec{h_{1}},\vec{h_{2}},...,\vec{h_{N}}\}, \\ \vec{h_{i}} \in \mathbb{R}^{F} \\ W \in \mathbb{R}^{F \times F^{'}} \\ e_{ij}=a(Wh_{i},Wh_{j}) \\ k \in \mathcal{N}_{i},\text{ neighbourhood nodes}\\ a_{ij}=\text{softmax}_{j}(e_{ij})=\frac{\exp(e_{ij})}{\sum_{k \in \mathcal{N}_{i}} \exp(e_{ik})}$$

Graph Convolutional Networks GCN

#machine learning #graph #GNN

$$H^{(l+1)}=\sigma(\tilde{D}^{-\frac{1}{2}}\tilde{A}\tilde{D}^{-\frac{1}{2}}H^{l}W^{l})\\ \tilde{A}=A+I_{N}\\ \tilde{D}_{ii}=\sum_{j}\tilde{A}_{ij} \\ H^{0}=X \\ \mathcal{L}=-\sum_{l \in Y}\sum^{F}_{f=1} Y_{lf} \ln Z_{lf}$$

Graph Laplacian

#machine learning #graph #GNN

$$L=I_{N}-D^{-\frac{1}{2}}AD^{-\frac{1}{2}} \\ L=U\Lambda U^{T}$$

GraphSage

#machine learning #graph #GNN

$$h^{0}_{v} \leftarrow x_{v} \\ \textbf{for} k \in \{1,2,...,K\} \text{do}\\ \textbf{for} v \in V \text{do} \\ h^{k}_{N_{v}} \leftarrow \textbf{AGGREGATE}_{k}(h^{k-1}_{u}, u \in N(v)); \\ h^{k}_{v} \leftarrow \sigma (W^{k} \textbf{concat}(h^{k-1}_{v},h^{k}_{N_{v}})) \\ \textbf{end} \\ h^{k}_{v}=h^{k}_{v}/||h^{k}_{v}||_{2},\forall v \in V \\ \textbf{end} \\ z_{v} \leftarrow h^{k}_{v} \\ J_{\textbf{z}_{u}}=-\log (\sigma (\textbf{z}_{u}^{T}\textbf{z}_{v})) - Q \mathbb{E}_{v_{n} \sim p_n(v)} \log(\sigma (-\textbf{z}_{u}^{T}\textbf{z}_{v_{n}}))$$

Mahalanobis Distance

#machine learning #mahalanobis

$$D_{M}(x,y)=\sqrt{(x-y)^{T}\Sigma^{-1}(x-y)}$$

Maximum Mean Discrepancy MMD

#machine learning #mmd

$$\textup{MMD}(\mathbb{F},X,Y):=\sup_{f \in\mathbb{F}}(\frac{1}{m}\sum_{i=1}^{m}f(x_{i}) -\frac{1}{n}\sum_{j=1}^{n}f(y_{j}))$$

Model-Agnostic Meta-Learning MAML

#machine learning #meta learning

$$\min_{\theta} \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}(f_{\theta^{'}_{i}}) = \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}(f_{\theta_{i} - \alpha \nabla_{\theta} \mathcal{L}_{\mathcal{T}_{i}} (f_{\theta}) })$$

Multi-Gate Mixture of Experts MMoE

#machine learning #multi task

$$g^{k}(x)=\text{softmax}(W_{gk}x) \\ f^{k}(x)=\sum^{n}_{i=1}g^{k}(x)_{i}f_{i}(x) \\ y_{k}=h^{k}(f^{k}(x))$$

Progressive Layered Extraction PLE

#machine learning #multi task

$$g^{k}(x)=w^{k}(x)S^{k}(x) \\ w^{k}(x)=\text{softmax}(W^{k}_{g}x) \\ S^{k}(x)=\[E^{T}_{(k,1)},E^{T}_{(k,2)},...,E^{T}_{(k,m_{k})},E^{T}_{(s,1)},E^{T}_{(s,2)},...,E^{T}_{(s,m_{s})}\]^{T} \\ y^{k}(x)=t^{k}(g^{k}(x)) \\ g^{k,j}(x)=w^{k,j}(g^{k,j-1}(x))S^{k,j}(x) $$

Prototypical Networks Protonets

#machine learning #meta learning

$$c_{k}=\frac{1}{|S_{k}|}\sum_{(x_{i},y_{i}) \in S_{k}} f_{\phi}(x) \\ p_{\phi}(y=k|x)=\frac{\exp(-d(f_{\phi}(x), c_{k}))}{\sum_{k^{'}} \exp(-d(f_{\phi}(x), c_{k^{'}})} \\\min J(\phi)=-\log p_{\phi}(y=k|x)$$

Proximal Policy Optimization PPO

#machine learning

$$L^{CLIP}(\theta)=E_{t}[\min(r_{t}(\theta))A_{t}, \text{clip}(r_{t}(\theta), 1-\epsilon,1+\epsilon)A_{t}]$$

Proximal Policy Optimization PPO

#machine learning #AI #LLM

$$\arg\max\limits_{\pi}{ E_{p \sim D,g \sim \pi} [R(g|p)] }, R(g|p) = \tilde{R}_{c}(g|p) - \beta D_{KL}( \pi_{\theta} (g|p) || \pi_{0} (g|p))$$

RotatE

#machine learning #KG

$$f_{r}(h, t) = ||h \circ r - t||^{2}_{2}$$

SME Linear

#machine learning #KG

$$\epsilon(lhs,rel,rhs)=E_{lhs(rel)}^{T}E_{rhs(rel)} \\=(W_{l1}E_{lhs}^{T} + W_{l2}E_{rel}^{T} + b_{l})^{T}(W_{r1}E_{rhs}^{T} + W_{r2}E_{rel}^{T} + b_{r})$$

SimplE

#machine learning #KG

$$s(e_{i}, r, e_{j}) = \frac{1}{2}( + )$$

TransE

#machine learning

$$\mathcal{L}=\sum_{(h,r,t) \in S} \sum_{(h^{'},r^{'},t^{'}) \in S^{'}_{(h,r,t)}} \[ \gamma + d(h + r, t) - d(h^{'} + r^{'}, t^{'}) \]_{+} \\ S^{'}_{(h,r,t)}=\{(h^{'},r,t)|h^{'} \in E\} \cup \{(h,r,t^{'})|t^{'} \in E\} \\ d(h + r, t)=||h + r - t||^{2}_{2}$$

TransH

#machine learning #KG

$$f_{r}(h,t) =||h_{\perp} + d_{r} - t_{\perp} ||^{2}_{2}=||(h - w_{r}hw_{r}) + d_{r} - (t - w_{r}tw_{r}) ||^{2}_{2}$$

TransR

#machine learning #KG

$$h_{r}=hM_{r}, t_{r}=tM_{r} \\f_{r}(h, t) = ||h_{r} + r - t_{r}||^{2}_{2}=||hM_{r}+r-tM_{r}||^{2}_{2}$$

Variational AutoEncoder VAE

#machine learning #VAE

$$\log p_{\theta}(x)=\mathbb{E}_{q_{\phi}(z|x)}[\log p_{\theta}(x)] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log \frac{p_{\theta}(x,z)}{p_{\theta}(z|x)}] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log [\frac{p_{\theta}(x,z)}{q_{\phi}(z|x)} \times \frac{q_{\phi}(z|x)}{p_{\theta}(z|x)}]] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log [\frac{p_{\theta}(x,z)}{q_{\phi}(z|x)} ]] +D_{KL}(q_{\phi}(z|x) || p_{\theta}(z|x))\\$$

X-Learner

#machine learning #causual inference

$$\tilde{D}^{1}_{i}:=Y^{1}_{i}-\hat{\mu}_{0}(X^{1}_{i}),\tilde{D}^{0}_{i}:=\hat{\mu}_{1}(X^{0}_{i})-Y^{0}_{i}\\ \hat{\tau}(x)=g(x)\hat{\tau}_{0}(x) + (1-g(x))\hat{\tau}_{1}(x)$$

Gamma Distribution

#Math #Statistics

Poisson Binomial Distribution

#Math #Statistics

$$Pr(K = k) = \sum_{A \in F_{k}} \prod_{i \in A} p_{i} \prod_{j \in A_{c}} (1-p_{j})$$

Student t-Distribution

#Math #Statistics

Chatbot close

Bot
Hi ,
How can I help you today?

Send

Navigation

Equation Database

economics

nlp

physics

ai

math

machine learning

statistics

EQUATION LIST

economics

nlp

physics

ai

math

machine learning

statistics