Equation Database

EQUATION LIST

economics

• Cox-Ingersoll-Ross CIR

  #Financial #Economics

  $$\mathrm{d} r(t) = a[b - r(t)] \mathrm{d} t + \sigma \sqrt{r(t)} \mathrm{d} Z(t) \\ P(r, t, T) = A(T-t)e^{-rB(T-t)} \\ \gamma = \sqrt{(a-\bar{\phi})^{2} + 2 \sigma^{2}} \\ q(r, t, T) = \sigma \sqrt{r} B(T-t) \\ \text{yield to maturity} \\ \frac{2ab}{ a - \bar{\phi} + \gamma}$$

  READ MORE

• Equation of Exchange

  #Economics #MacroEconomics

  $$\text{MV} = \text{PQ}$$

  READ MORE

• Geometric Brownian Motion SDEs

  #Financial #Economics

  $$\mathrm{d}Y(t) = \mu Y(t)dt + \sigma Y(t) \mathrm{d}Z(t) \\ \mathrm{d}[\ln Y(t)] = (\mu - \frac{\sigma^2}{2}) \mathrm{d}t + \sigma \mathrm{d}Z(t) \\ Y(t) = T(0) e^{(\mu - \frac{\sigma^2}{2})t + \sigma Z(t)}$$

  READ MORE

• Ito Lemma

  #Financial #Economics

  $$\mathrm{d}X(t) = a(t, X(t)) \mathrm{d}t + b(t, X(t))\mathrm{d} Z(t) \\ Y(t) = f(t, X(t)) \mathrm{d}t \\ \mathrm{d} Y(t) = f_{t}(t, X(t)) + f_{x}(t, X(t))\mathrm{d} X(t) + \frac{1}{2} f_{xx}(t, X(t))[\mathrm{d}X(t)]^{2} \\ [\mathrm{d} X(t)]^{2} = b^{2}(t, X(t))\mathrm{d} t$$

  READ MORE

• Risk-Neutral Valuation and Power Contracts

  #Financial #Economics

  $$\frac{\mathrm{d}S(t)}{S(t)} = (r - \delta) \mathrm{d}t + \sigma \mathrm{d} \tilt{Z}(t) \\ \tilt{Z}(t) = Z(t) + \phi t \\ V(S(t), t) = e^{-r(T-t)} E^{*}[V(S(T), T) | S(T)] \\ F^{p}_{t, T}(S^{a}) = S^{a}(t) e ^{ (-r + a(r-\delta) + \frac{1}{2} a(a-1)\sigma^{2})(T-t)}$$

  READ MORE

• Sharpe Ratio

  #Financial #Economics

  $$\frac{\mathrm{d}X(t)}{X(t)} = m \mathrm{d}t + s \mathrm{d}Z(t) \\ \phi = \frac{m + \delta -r }{s} \\ \phi = \frac{a - r}{\sigma}$$

  READ MORE

• Stock Prices as Geometric Brownian Motion

  #Financial #Economics

  $$\frac{\mathrm{d}S(t)}{S(t)} = (a - \delta) \mathrm{d}t + \sigma \mathrm{d}Z(t) \\ S(t) = S(0) e^{(a - \delta - \frac{\sigma^{2}}{2})t + \sigma Z(t)} \\ \mathrm{d}[\ln S(t)] = (a - \delta - \frac{\sigma^{2}}{2}) \mathrm{d}t + \sigma \sigma \mathrm{d} Z(t) \\ S(t) \sim \ln( \ln S(0) + (a - \delta - \frac{\sigma^2}{2})t, \sigma^{2}t)$$

  READ MORE

nlp

• Direct Policy Optimization DPO

  #nlp #llm #RLHF

  $$\pi_{r} (y|x) = \frac{1}{Z(x)} \pi_{ref} (y|x) \exp(\frac{1}{\beta} r(x,y) ) , r(x,y) = \beta \log \frac{\pi_{r} (y|x)}{\pi_{ref} (y|x)} + \beta \log Z(x) , p^{*}(y_{1} > y_{2} |x) = \frac{1}{1+\exp{(\beta \frac{\pi^{*} (y_{2}|x)}{\pi_{ref} (y_{2}|x)} - \beta \frac{\pi^{*} (y_{1}|x)}{\pi_{ref} (y_{1}|x)} )}} , \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = -\mathbb{E}_{(x, y_{w},y_{l}) \sim D } [\log \sigma (\beta \frac{\pi_{\theta} (y_{w}|x)}{\pi_{ref} (y_{w}|x)} - \beta \frac{\pi_{\theta} (y_{l}|x)}{\pi_{ref} (y_{l}|x)} )] , \nabla \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = - \beta \mathbb{E}_{(x, y_{w},y_{l}) \sim D } [ \sigma ( \hat{r}_{\theta} (x, y_{l}) - \hat{r}_{\theta} (x, y_{w})) [\nabla_{\theta} \log \pi (y_{w}|x) - \nabla_{\theta} \log \pi (y_{l}|x) ] ] , \hat{r}_{\theta} (x, y) = \beta \log (\frac{\pi_{\theta} (y|x)}{\pi_{ref} (y|x)})$$

  READ MORE

• KTO Kahneman-Tversky Optimisation Equation

  #nlp #llm #AI

  $$f(\pi_\theta, \pi_\text{ref}) = \mathbb{E}_{x,y\sim\mathcal{D}}[ a_{x,y} v(r_\theta(x,y) - \mathbb{E}_{Q}[r_\theta(x, y')])] + C_\mathcal{D}$$

  READ MORE

• LOW RANK ADAPTATION LORA

  #nlp #llm #RLHF

  $$W_{0} + \Delta W_{0} = W_{0} + BA, h=W_{0}x + \Delta W_{0}x = W_{0}x + BAx, \text{Initialization:} A \sim N(0, \sigma^{2}), B = 0$$

  READ MORE

• RLHF Reinforcement Learning from Human Feedback

  #nlp #LLM #equation

  $$p^*(y_w \succ y_l|x) = \sigma(r^*(x,y_w) - r^*(x,y_l)) $$ $$ \mathcal{L}_R(r_\phi) = \mathbb{E}_{x,y_w,y_l \sim D}[- \log \sigma(r_\phi(x, y_w) - r_\phi(x, y_l))] $$ $$ \mathbb{E}_{x \in D, y \in \pi_\theta} [r_\phi(x,y)] - \beta D_{\text{KL}}(\pi_\theta(y|x) \| \pi_{\text{ref}}(y|x)) $$

  READ MORE

physics

• Electric Oscillations

  #physics #oscillations

  $$\text{Impedance} \\ Z=R+ix \\ \text{Series connection} \\ V=IZ, Z_{\rm tot}=\sum_i Z_i~,~~L_{\rm tot}=\sum_i L_i~,~~ \frac{1}{C_{\rm tot}}=\sum_i\frac{1}{C_i}~,~~Q=\frac{Z_0}{R}~,~~ Z=R(1+iQ\delta) \\ \text{Parallel connection} \\ \frac{1}{Z_{\rm tot}}=\sum_i\frac{1}{Z_i}~,~~ \frac{1}{L_{\rm tot}}=\sum_i\frac{1}{L_i}~,~~ C_{\rm tot}=\sum_i C_i~,~~Q=\frac{R}{Z_0}~,~~ Z=\frac{R}{1+iQ\delta}$$

  READ MORE

• General Relativity

  #physics #relativity

  $$\frac{\mathrm{d}^{2} x^{\alpha}}{\mathrm{d} s^{2}}+\Gamma^{\alpha}_{\beta\gamma}\frac{\mathrm{d} x^{\beta}}{\mathrm{d} s}\frac{\mathrm{d} x^{\gamma}}{\mathrm{d} s}=0$$

  READ MORE

• Maxwell Equations Integral

  #physics #maxwell #electricity #magnetism

  $$\oiint (\vec{D}\cdot \vec{n}) \mathrm{d}^{2}A=Q_{\text{free,included}}\\ \oiint (\vec{B}\cdot \vec{n}) \mathrm{d}^{2}A=0 \\ \oint \vec{E} \mathrm{d}\vec{s}=-\frac{\mathrm{d}\Phi}{\mathrm{d}t}\\ \oint \vec{H} \mathrm{d}\vec{s}=I_{\text{free,included}}+\frac{\mathrm{d}\Psi }{\mathrm{d}t}$$

  READ MORE

• Mechanic Oscillations

  #physics #mechanic #oscillations

  $$m\ddot{x}=F(t)-k\dot{x}-Cx \\ F(t)=\hat{F}\cos(\omega t) \\ -m\omega^2 x=F-Cx-ik\omega x \\ \omega_0^2=C/m \\ x=\frac{F}{m(\omega_0^2-\omega^2)+ik\omega} \\ \dot{x}=\frac{F}{i\sqrt{Cm}\delta+k} \\ \delta=\frac{\omega}{\omega_0}-\frac{\omega_0}{\omega} \\ Z=F/\dot{x} \\ Q=\frac{\sqrt{Cm}}{k}$$

  READ MORE

• Waves In Long Conductors

  #physics #waves #oscillations

  $$Z_0=\sqrt{\frac{dL}{dx}\frac{dx}{dC}} \\ v=\sqrt{\frac{dx}{dL}\frac{dx}{dC}}$$

  READ MORE

math

• Arithmetic and Geometric Progressions

  #math #arithmetic #geometric #progressions

  $$S_{n}=a+(a+d)+(a+2d)+...+[a+(n-1)d]=\frac{n}{2}[2a+(n-1)d] \\ S_{n}=a+ar+ar^{2}+...+ar^{n-1}=a\frac{1-r^{n}}{1-r}$$

  READ MORE

• Bessel Equation

  #math #bessel equation

  $$x^{2}\frac{\mathrm{d}^{2} y}{\mathrm{d} x^{2}}+x\frac{\mathrm{d} y}{\mathrm{d} x}+(x^{2}-m^{2})y=0 \\ J_{m}(x)=\sum^{\infty}_{k=0}\frac{(-1)^{k} (x/2)^{m+2k}}{k!(m+k)!}$$

  READ MORE

• Binomial Expansion

  #math #binomial #expansion

  $$(1+x)^{n}=1+nx+\frac{n(n-1)}{2!}x^{2}+...+ C^{r}_{n}x^{r} +...+x^{n} \\ C^{r}_{n}=\frac{n!}{r!(n-r)!}$$

  READ MORE

• Complex Numbers

  #math #complex numbers

  $$z=x+iy=r(\cos \theta + i \sin \theta)=re^{i(\theta+2n\pi)} \\ z^{*}=x-iy=r(\cos \theta - i \sin \theta)=re^{-i\theta} \\ zz^{*}=|z|^{2}=x^{2}+y^{2}$$

  READ MORE

• Convergence of Series

  #math #convergence #series

  $$S_{n}=u_{1}+u_{2}+...+u_{n} \\ \text{Converge AS } n \rightarrow \infty \text{, If} \lim_{n \rightarrow \infty} |\frac{u_{n+1}}{u_{n}}| < 1$$

  READ MORE

• De Moivre's Theorem

  #math #complex number #De Moivre

  $$(\cos \theta + i \sin \theta)^{n}=e^{in\theta}=\cos n\theta + i \sin n\theta$$

  READ MORE

• Determinants of a Matrix

  #math #determinants

  $$|A|=\sum_{i,j,k,...}\epsilon_{ijk}A_{1i}A_{2j}A_{3k}...$$

  READ MORE

• Diffusion Conduction Equation

  #math #diffusion #conduction

  $$\kappa u_{xx} = u_{t} \\ \frac{\partial \Psi}{\partial t}=\kappa \triangledown^{2} \Psi$$

  READ MORE

• Eigenvalues and Eigenvectors

  #math #eigenvalues #eigenvectors

  $$A\mathbf{u}=\lambda\mathbf{u} \\ P_{n}(\lambda)=|A-\lambda I|$$

  READ MORE

• Exponential Distribution

  #Math #Statistics

  $$f(x, \lambda)=\lambda e^{-\lambda x} \\ F(x, \lambda)=1 - e^{-\lambda x}$$

  READ MORE

• Fisher Skewness

  #Math #Statistics

  $$\gamma_1 = \frac{{\mu_3 }}{{\mu_2 ^{{3 \mathord{\left/ {\vphantom {3 2}} \right. \kern-\nulldelimiterspace} 2}} }} = \frac{{\mu_3 }}{{\sigma ^3 }}$$

  READ MORE

• Fourier Series

  #math #fourier series

  $$y(x)=c_{0}+\sum^{M}_{m=1}c_{m}\cos mx+\sum^{M^{'}}_{m=1}s_{m}\sin mx \\ c_{0}=\frac{1}{2\pi}\int^{\pi}_{-\pi}y(x) \mathrm{d} x \\ c_{m}=\frac{1}{\pi}\int^{\pi}_{-\pi}y(x) \cos mx \mathrm{d} x \\ s_{m}=\frac{1}{\pi}\int^{\pi}_{-\pi}y(x) \sin mx \mathrm{d} x$$

  READ MORE

• Fourier Transforms

  #math #fourier transforms

  $$y(t)=\frac{1}{2\pi}\int_{-\infty}^{\infty} \hat{y}(\omega) e^{i\omega t} \mathrm{d} \omega \\ \hat{y}(\omega)=\int_{-\infty}^{\infty} y(t) e^{-i\omega t} \mathrm{d} t$$

  READ MORE

• Gamma Distribution

  #Math #Statistics

  $$\Gamma \left( a \right) = \int\limits_0^\infty {s^{a - 1} } e^{ - s} ds \\ P(x) = \frac{x^{\alpha-1} e^{-frac{x}{\theta}}}{\Gamma(\alpha) \theta^{\alpha}} \\ \mu = \alpha \theta \\ \sigma^{2} = \alpha \theta^{2} \\ \gamma_{1} = \frac{2}{\sqrt{\alpha}} \\ \gamma_{2} = \frac{6}{\alpha}$$

  READ MORE

• Heat Equation

  #math #heat equation

  $$u_{t}={\alpha}^{2}u_{xx}$$

  READ MORE

• Laplace Equation

  #math #laplace equation

  $$u_{xx} = 0$$

  READ MORE

• Laplace Transform

  #math #laplace transform

  $$\bar{y}(s)=\mathcal{L}(y(t))=\int^{\infty}_{0} e^{-st} y(t) \mathrm{d}t$$

  READ MORE

• Legendre Equation

  #math #legendre equation

  $$(1-x^{2})\frac{\mathrm{d}^{2} y}{\mathrm{d} x^{2}}-2x\frac{\mathrm{d} y}{\mathrm{d} x}+l(l+1)y=0 \\ P_{l}(x)=\frac{1}{2^{l}l!}(\frac{\mathrm{d}}{\mathrm{d} x})^{l}(x^2-1)^{l}\\ P_{l}(x)=\frac{1}{l}[(2l-1)xP_{l-1}(x)-(l-1)P_{l-2}(x)]$$

  READ MORE

• Maclaurin Series

  #math #maclaurin series

  $$y(x)=y(a+u)=y(0)+x\frac{\mathrm{d} y}{\mathrm{d} x}+\frac{1}{2!}x^{2}\frac{\mathrm{d}^2 y}{\mathrm{d} x^{2}}+\frac{1}{3!}x^{3}\frac{\mathrm{d}^3 y}{\mathrm{d} x^{3}}+...$$

  READ MORE

• Normal Gaussian Distribution

  #Math #Statistics

  $$X \sim \mathcal{N}(\mu,\sigma^2) \\ f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp{[-\frac{(x-\mu)^{2}}{2\sigma^{2}}]}$$

  READ MORE

• Plane Wave Expansion

  #math #plane wave expansion

  $$\exp(ikz)=\exp(ikr\cos \theta=\sum^{\infty}_{l=0}(2l+1)i^{l}j_{l}(kr)P_{l}(\cos \theta)$$

  READ MORE

• Power Series for Complex Variables

  #math #complex variables

  $$e^{z}=1+z+\frac{z^{2}}{2!}+\frac{z^{3}}{3!}+...+\frac{z^{n}}{n!}+...\\ \sin z=z-\frac{z^{3}}{3!}+\frac{z^{5}}{5!}-...\\ \cos z=1-\frac{z^{2}}{2!}+\frac{z^{4}}{4!}-...\\ \ln (1+z)=1-\frac{z^{2}}{2!}+\frac{z^{3}}{3!}-...\\ (1+z)^{n}=1+nz+\frac{n(n-1)}{2!}z^{2}+\frac{n(n-1)(n-2)}{3!}z^{3}+...$$

  READ MORE

• Power Series with Real Variables

  #math #power #series

  $$e^{x}=1+x+\frac{x^{2}}{2!}+...+\frac{x^{n}}{n!}+... \\ \ln(1+x) = x - \frac{x^{2}}{2} + \frac{x^{3}}{3} + ... + (-1)^{n+1}\frac{x^{n}}{n!} +... \\ \cos(x) = \frac{e^{ix}+e^{-ix}}{2}=1-\frac{x^{2}}{2!}+\frac{x^{4}}{4!}-\frac{x^{6}}{6!}+...\\ \sin(x) = \frac{e^{ix}-e^{-ix}}{2i}=x-\frac{x^{3}}{3!}+\frac{x^{5}}{5!}+...$$

  READ MORE

• Regression Least Squares Fitting

  #math #regression

  $$y_{i}=\alpha + \beta (x_{i} - \bar{x}) + \epsilon_{i} \\ \hat{\alpha}=\bar{y} \\ \hat{\beta}=\frac{s^{2}_{xy}}{s^{2}_{x}}$$

  READ MORE

• Spherical Harmonics Equation

  #math #spherical harmonics

  $$[\frac{1}{\sin \theta} \frac{\partial}{\partial \theta}(\sin \theta \frac{\partial}{\partial \theta}) + \frac{1}{\sin^{2} \theta} \frac{\partial^{2}}{\partial \phi^{2}}) ] Y^{m}_{l} + l(l+1) Y^{m}_{l}=0 \\ Y^{m}_{l}(\theta,\phi)=\sqrt{\frac{2l+1}{4 \pi} \frac{(l-|m|)!}{(l+|m|)!}}P^{m}_{l}(\cos \theta) e^{im \phi} \times \begin{cases}(-1)^{m} & m\ge 0 \\ 1 & m <0 \end{cases}$$

  READ MORE

• Taylor Series

  #math #taylor series

  $$y(x)=y(a+u)=y(a)+u\frac{\mathrm{d} y}{\mathrm{d} x}+\frac{1}{2!}u^{2}\frac{\mathrm{d}^2 y}{\mathrm{d} x^{2}}+\frac{1}{3!}u^{3}\frac{\mathrm{d}^3 y}{\mathrm{d} x^{3}}+...$$

  READ MORE

• Wave Equation

  #math #wave equation

  $$u_{tt}=c^{2}u_{xx}$$

  READ MORE

• gaussian process

  #math #gaussian process

  $$\log p(y|X) \propto -[y^{T}(K + \sigma^{2}I)^{-1}y+\log|K + \sigma^{2}I|] \\ f(X)=\[f(x_{1}),f(x_{2}),...,f(x_{N}))\]^{T} \sim \mathcal{N}(\mu, K_{X,X}) \\ f_{*}|X_{*},X,y \sim \mathcal{N}(\mathbb{E}(f_{*}),\text{cov}(f_{*})) \\ \text{cov}(f_{*})=K_{X_{*},X_{*}}-K_{X_{*},X}[K_{X,X}+\sigma^{2}I]^{-1}K_{X,X_{*}}$$

  READ MORE

machine learning

• Average Treatment Effect ATE

  #machine learning #causual inference

  $$\text{ATE}:=\mathbb{E}[Y(1)-Y(0)]$$

  READ MORE

• Bound on Target Domain Error

  #machine learning #transfer learning

  $$\epsilon_{T}(h) \le \hat{\epsilon}_{S}(h) + \sqrt{\frac{4}{m}(d \log \frac{2em}{d} + \log \frac{4}{\delta })} + d_{\mathcal{H}}(\tilde{\mathcal{D}}_{S}, \tilde{\mathcal{D}}_{T}) + \lambda \\ \lambda = \lambda_{S} + \lambda_{T}$$

  READ MORE

• Conditional Average Treatment Effect CATE

  #machine learning #causual inference

  $$\tau(x):=\mathbb{E}[Y(1)-Y(0)|X=x]$$

  READ MORE

• Diffusion Model Forward Process

  #machine learning #diffusion

  $$q(x_{t}|x_{t-1})=\mathcal{N}(x_{t};\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}I) \\q(x_{1:T}|x_{0})=\prod_{t=1}^{T}q(x_{t}|x_{t-1})$$

  READ MORE

• Diffusion Model Reverse Process

  #machine learning #diffusion

  $$p_\theta(\mathbf{x}_{0:T}) = p(\mathbf{x}_T) \prod^T_{t=1} p_\theta(\mathbf{x}_{t-1} \vert \mathbf{x}_t) \\ p_\theta(\mathbf{x}_{t-1} \vert \mathbf{x}_t) = \mathcal{N}(\mathbf{x}_{t-1}; \boldsymbol{\mu}_\theta(\mathbf{x}_t, t), \boldsymbol{\Sigma}_\theta(\mathbf{x}_t, t))$$

  READ MORE

• Diffusion Model Variational Lower Bound

  #machine learning #diffusion

  $$\begin{aligned} - \log p_\theta(\mathbf{x}_0) &\leq - \log p_\theta(\mathbf{x}_0) + D_\text{KL}(q(\mathbf{x}_{1:T}\vert\mathbf{x}_0) \| p_\theta(\mathbf{x}_{1:T}\vert\mathbf{x}_0) ) \\ &= -\log p_\theta(\mathbf{x}_0) + \mathbb{E}_{\mathbf{x}_{1:T}\sim q(\mathbf{x}_{1:T} \vert \mathbf{x}_0)} \Big[ \log\frac{q(\mathbf{x}_{1:T}\vert\mathbf{x}_0)}{p_\theta(\mathbf{x}_{0:T}) / p_\theta(\mathbf{x}_0)} \Big] \\ &= -\log p_\theta(\mathbf{x}_0) + \mathbb{E}_q \Big[ \log\frac{q(\mathbf{x}_{1:T}\vert\mathbf{x}_0)}{p_\theta(\mathbf{x}_{0:T})} + \log p_\theta(\mathbf{x}_0) \Big] \\ &= \mathbb{E}_q \Big[ \log \frac{q(\mathbf{x}_{1:T}\vert\mathbf{x}_0)}{p_\theta(\mathbf{x}_{0:T})} \Big] \\ \text{Let }L_\text{VLB} &= \mathbb{E}_{q(\mathbf{x}_{0:T})} \Big[ \log \frac{q(\mathbf{x}_{1:T}\vert\mathbf{x}_0)}{p_\theta(\mathbf{x}_{0:T})} \Big] \geq - \mathbb{E}_{q(\mathbf{x}_0)} \log p_\theta(\mathbf{x}_0) \end{aligned}$$

  READ MORE

• Diffusion Model Variational Lower Bound Loss

  #machine learning #diffusion

  $$\begin{aligned} L_\text{VLB} &= L_T + L_{T-1} + \dots + L_0 \\ \text{where } L_T &= D_\text{KL}(q(\mathbf{x}_T \vert \mathbf{x}_0) \parallel p_\theta(\mathbf{x}_T)) \\ L_t &= D_\text{KL}(q(\mathbf{x}_t \vert \mathbf{x}_{t+1}, \mathbf{x}_0) \parallel p_\theta(\mathbf{x}_t \vert\mathbf{x}_{t+1})) \text{ for }1 \leq t \leq T-1 \\ L_0 &= - \log p_\theta(\mathbf{x}_0 \vert \mathbf{x}_1) \end{aligned}$$

  READ MORE

• Domain Adaptation H-Divergence

  #machine learning #transfer learning

  $$d_{\mathcal{H}}(\mathcal{D},\mathcal{D}^{'})=2\sup_{h \in \mathcal{H}}|\Pr_{\mathcal{D}}[I(h)]-\Pr_{\mathcal{D}^{'}}[I(h)]|$$

  READ MORE

• Domain-Adversarial Neural Networks DANN

  #machine learning #transfer learning

  $$\min [\frac{1}{m}\sum^{m}_{1}\mathcal{L}(f(\textbf{x}^{s}_{i}),y_{i})+\lambda \max(-\frac{1}{m}\sum^{m}_{i=1}\mathcal{L}^{d}(o(\textbf{x}^{s}_{i}),1)-\frac{1}{m^{'}}\sum^{m^{'}}_{i=1}\mathcal{L}^{d}(o(\textbf{x}^{t}_{i}),0))]$$

  READ MORE

• Generative Adversarial Networks GAN

  #machine learning #gan

  $$\min_{G} \max_{D} V(D,G)=\mathbb{E}_{x \sim p_{data}(x)}[\log D(x)]+\mathbb{E}_{z \sim p_{z}(z)}[\log(1-D(G(z)))]$$

  READ MORE

• Graph Attention Network GAT

  #machine learning #graph #GNN

  $$h=\{\vec{h_{1}},\vec{h_{2}},...,\vec{h_{N}}\}, \\ \vec{h_{i}} \in \mathbb{R}^{F} \\ W \in \mathbb{R}^{F \times F^{'}} \\ e_{ij}=a(Wh_{i},Wh_{j}) \\ k \in \mathcal{N}_{i},\text{ neighbourhood nodes}\\ a_{ij}=\text{softmax}_{j}(e_{ij})=\frac{\exp(e_{ij})}{\sum_{k \in \mathcal{N}_{i}} \exp(e_{ik})}$$

  READ MORE

• Graph Convolutional Networks GCN

  #machine learning #graph #GNN

  $$H^{(l+1)}=\sigma(\tilde{D}^{-\frac{1}{2}}\tilde{A}\tilde{D}^{-\frac{1}{2}}H^{l}W^{l})\\ \tilde{A}=A+I_{N}\\ \tilde{D}_{ii}=\sum_{j}\tilde{A}_{ij} \\ H^{0}=X \\ \mathcal{L}=-\sum_{l \in Y}\sum^{F}_{f=1} Y_{lf} \ln Z_{lf}$$

  READ MORE

• Graph Laplacian

  #machine learning #graph #GNN

  $$L=I_{N}-D^{-\frac{1}{2}}AD^{-\frac{1}{2}} \\ L=U\Lambda U^{T}$$

  READ MORE

• GraphSage

  #machine learning #graph #GNN

  $$h^{0}_{v} \leftarrow x_{v} \\ \textbf{for} k \in \{1,2,...,K\} \text{do}\\ \textbf{for} v \in V \text{do} \\ h^{k}_{N_{v}} \leftarrow \textbf{AGGREGATE}_{k}(h^{k-1}_{u}, u \in N(v)); \\ h^{k}_{v} \leftarrow \sigma (W^{k} \textbf{concat}(h^{k-1}_{v},h^{k}_{N_{v}})) \\ \textbf{end} \\ h^{k}_{v}=h^{k}_{v}/||h^{k}_{v}||_{2},\forall v \in V \\ \textbf{end} \\ z_{v} \leftarrow h^{k}_{v} \\ J_{\textbf{z}_{u}}=-\log (\sigma (\textbf{z}_{u}^{T}\textbf{z}_{v})) - Q \mathbb{E}_{v_{n} \sim p_n(v)} \log(\sigma (-\textbf{z}_{u}^{T}\textbf{z}_{v_{n}}))$$

  READ MORE

• Maximum Mean Discrepancy MMD

  #machine learning #mmd

  $$\textup{MMD}(\mathbb{F},X,Y):=\sup_{f \in\mathbb{F}}(\frac{1}{m}\sum_{i=1}^{m}f(x_{i}) -\frac{1}{n}\sum_{j=1}^{n}f(y_{j}))$$

  READ MORE

• Model-Agnostic Meta-Learning MAML

  #machine learning #meta learning

  $$\min_{\theta} \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}(f_{\theta^{'}_{i}}) = \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}(f_{\theta_{i} - \alpha \nabla_{\theta} \mathcal{L}_{\mathcal{T}_{i}} (f_{\theta}) })$$

  READ MORE

• S-Learner

  #machine learning #causual inference

  $$\mu(x,w)=\mathbb{E}[Y_{i}|X=x_{i},W=w] \\ \hat{\tau}(x)=\hat{\mu}(x,1)-\hat{\mu}(x,0)$$

  READ MORE

• T-Learner

  #machine learning #causual inference

  $$\mu_{0}(x)=\mathbb{E}[Y(0)|X=x],\mu_{1}(x)=\mathbb{E}[Y(1)|X=x],\\ \hat{\tau}(x)=\hat{\mu}_{1}(x)-\hat{\mu}_{0}(x)$$

  READ MORE

• TransE

  #machine learning

  $$\mathcal{L}=\sum_{(h,r,t) \in S} \sum_{(h^{'},r^{'},t^{'}) \in S^{'}_{(h,r,t)}} \[ \gamma + d(h + r, t) - d(h^{'} + r^{'}, t^{'}) \]_{+} \\ S^{'}_{(h,r,t)}=\{(h^{'},r,t)|h^{'} \in E\} \cup \{(h,r,t^{'})|t^{'} \in E\} \\ d(h + r, t)=||h + r - t||^{2}_{2}$$

  READ MORE

• Variational AutoEncoder VAE

  #machine learning #VAE

  $$\log p_{\theta}(x)=\mathbb{E}_{q_{\phi}(z|x)}[\log p_{\theta}(x)] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log \frac{p_{\theta}(x,z)}{p_{\theta}(z|x)}] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log [\frac{p_{\theta}(x,z)}{q_{\phi}(z|x)} \times \frac{q_{\phi}(z|x)}{p_{\theta}(z|x)}]] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log [\frac{p_{\theta}(x,z)}{q_{\phi}(z|x)} ]] +D_{KL}(q_{\phi}(z|x) || p_{\theta}(z|x))\\$$

  READ MORE

statistics

• Exponential Distribution

  #Math #Statistics

  $$f(x, \lambda)=\lambda e^{-\lambda x} \\ F(x, \lambda)=1 - e^{-\lambda x}$$

  READ MORE

• Fisher Skewness

  #Math #Statistics

  $$\gamma_1 = \frac{{\mu_3 }}{{\mu_2 ^{{3 \mathord{\left/ {\vphantom {3 2}} \right. \kern-\nulldelimiterspace} 2}} }} = \frac{{\mu_3 }}{{\sigma ^3 }}$$

  READ MORE

• Gamma Distribution

  #Math #Statistics

  $$\Gamma \left( a \right) = \int\limits_0^\infty {s^{a - 1} } e^{ - s} ds \\ P(x) = \frac{x^{\alpha-1} e^{-frac{x}{\theta}}}{\Gamma(\alpha) \theta^{\alpha}} \\ \mu = \alpha \theta \\ \sigma^{2} = \alpha \theta^{2} \\ \gamma_{1} = \frac{2}{\sqrt{\alpha}} \\ \gamma_{2} = \frac{6}{\alpha}$$

  READ MORE

• Normal Gaussian Distribution

  #Math #Statistics

  $$X \sim \mathcal{N}(\mu,\sigma^2) \\ f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp{[-\frac{(x-\mu)^{2}}{2\sigma^{2}}]}$$

  READ MORE