🔬 Taylor's Theorem
Author: MathClown
Theorem. Given a continuously differentiable function $ f: \mathbb{R}^n \rightarrow \mathbb{R} $, and given $ x, p \in \mathbb{R}^n $, we have that
\[f(x+p) = f(x) + \int_{0}^{1} \nabla f(x+\gamma p)^T p \, d\gamma, \quad \tag{1}
\]
\[f(x+p) = f(x) + \nabla f(x+\gamma p)^T p, \quad \text{some} \, \gamma \in (0,1).\tag{2}
\]
If $ f $ is twice continuously differentiable, we have
\[\nabla f(x+p) = \nabla f(x) + \int_{0}^{1} \nabla^2 f(x+\gamma p) p \, d\gamma, \tag{3}
\]
\[f(x+p) = f(x) + \nabla f(x)^T p + \frac{1}{2} p^T \nabla^2 f(x+\gamma p) p, \quad \text{some} \, \gamma \in (0,1). \quad \tag{4}
\]
(We sometimes call the relation (1) the "integral form" and (2) the "mean-value form" of Taylor's theorem.)
A consequence of (2) is that for \(f\) continuously differentiable at \(x\), we have
\[f(x+p) = f(x) + \nabla f(x)^T p + o(\|p\|). \tag{5}
\]