All notes
Calculus

Basics

Definitions of limit

($\epsilon$, $\delta$)-definition of limit

Wikipedia: definition of limit.

Infinitesimal definition

$$ \lim_{x\to a} f(x) = L $$ if and only if whenever the difference $x−a$ is infinitesimal, the difference $f(x)−L$ is infinitesimal, as well, or in formulas: if $st(x) = a$ then $st(f(x)) = L$

infinitesimal
it is smaller than any real number, yet it is greater than zero.
Hyperreal
Hyperreal $*\mathbf{R}$ is the extended $\mathbf{R}$ plus infinitesimal and infinities, without changing any of the elementary axioms of algebra.
Start part function
In non-standard analysis, the standard part function is a function from the limited (finite) hyperreal numbers to the real numbers.
non-standard calculus
is the modern application of infinitesimals, in the sense of non-standard analysis, to differential and integral calculus.

Continuity

Derivative

The derivative of the function $y = f(x)$: $$ \frac{dy}{dx} := \lim_{\Delta x \rightarrow 0} \frac{\Delta y}{\Delta x} = \lim_{\Delta x \rightarrow 0} \frac{f(x+\Delta x) - f(x)}{\Delta x}$$

It is the process turning secant lines into the tangent lines. Secant to Tangent.

Power rule

$$ \frac{d x^p}{dx} := \lim_{\Delta x \rightarrow 0} \frac{(x+\Delta x)^p - x^p}{\Delta x} = \lim_{\Delta x \rightarrow 0} x^{p-1} p + \binom{p}{2} x^{p-2} \Delta x + \binom{p}{3} x^{p-3} \Delta x^2 + \ldots$$ $$ = x^{p-1} p $$

Exponential rule

$$ {\frac {d}{dx}}a^x = a^x {\ln a} $$ $$ {\frac {d}{dx}}e^{ax} = ae^{ax} $$

Natural logrithm, $e$

mathworld: e.

$e$ is defined as by limit: $$ e :=\quad \lim _{n\to \infty }\left(1+{\frac {1}{n}}\right)^{n} $$

or by the infinite series: $$ e := \sum_{k=0}^\infty 1/(k!) $$ as first published by Newton (1669; reprinted in Whiteside 1968, p. 225).

And $e$ is the unique number with the property that the area of the region bounded by the hyperbola y=1/x, the x-axis, and the vertical lines x=1 and x=e is 1. In other words, $$ int_1^e(dx)/x=ln e=1 $$

Now we find the derivative of $\ln x$ using the formal definition of the derivative: $$ {\frac {d}{dx}}\ln(x)=\quad \lim _{\Delta x\to 0}{\frac {\ln(x+\Delta x)-\ln(x)}{\Delta x}}=$$ $$ \quad \lim _{\Delta x\to 0}{\frac {1}{\Delta x}}\ln \left({\frac {x+\Delta x}{x}}\right)=\quad \lim _{\Delta x\to 0}\ln \left({\frac {x+\Delta x}{x}}\right)^{\frac {1}{\Delta x}}$$

Let $n={\frac {x}{\Delta x}}$. Note that as $n$ approaches $\infty$ , $\Delta x$ approaches 0. So we can redefine our limit as: $$ \quad \lim _{n\to \infty }\ln \left(1+{\frac {1}{n}}\right)^{\frac {n}{x}}={\frac {1}{x}}\ln \left(\lim _{n\to \infty }\left(1+{\frac {1}{n}}\right)^{n}\right)={\frac {1}{x}}\ln(e)={\frac {1}{x}} $$

Logarithm Rule

wikibooks.org: derivatives of exponential and logarithm functions. $$ {\frac {d}{dx}}\ln(x)=\quad \lim _{\Delta x\to 0}{\frac {\ln(x+\Delta x)-\ln(x)}{\Delta x}} = $$ $$ \quad \lim _{\Delta x\to 0}{\frac {1}{\Delta x}}\ln \left({\frac {x+\Delta x}{x}}\right)$$

Let $ n={\frac {x}{\Delta x}} $. Note that as $n$ approaches $\infty$, $\Delta x$ approaches 0. So we can redefine our limit as:

$$ \quad \lim _{n\to \infty }\ln \left(1+{\frac {1}{n}}\right)^{\frac {n}{x}}={\frac {1}{x}}\ln \left(\lim _{n\to \infty }\left(1+{\frac {1}{n}}\right)^{n}\right)={\frac {1}{x}}\ln(e)={\frac {1}{x}} $$

$$ {\frac {d}{dx}}\log_b(x) = \frac{1}{x} \frac{1}{\ln b} $$

Properties of Derivatives

$$ \frac{d}{dx} cf(x) = c \frac{df}{dx} $$ $$ [cf(x)]' = cf'(x) $$

Sum and Difference Rules $$ \frac{d}{dx} [f(x) \pm g(x)] = \frac{df}{dx} \pm \frac{dg}{dx} $$ $$ [f(x) \pm g(x)]' = f'(x) \pm g'(x)$$

Other

$$ \frac{d}{dx} \left( \frac{1}{1-x} \right) = \frac{-1}{(1-x)^2} \cdot \frac{d}{dx} (1-x) = \frac{1}{(1-x)^2}$$

Taylor Series

Wolfram: TaylorSeries.

Taylor Series
A series expansion of a function about a point.
Maclaurin Series
Taylor series with $a=0$.

Taylor Series: $$ f(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f^{(3)}(a)}{3!}(x-a)^3 + \ldots $$ $$ f(x) = \sum_{i=0}^{\infty} \frac{f^{i}(a)}{i!}(x-a)^i $$

Taylor's theorem

Any function satisfying certain conditions can be expressed as a Taylor Series.

Lagrange Multiplier

Wolfram.

Lamar. Method of Lagrange Multipliers

Exams. See the ref before for answer.

A good explanation

Wikipedia. See good diagrams in the reference.

Integral Transform

Laplace Transform

拉普拉斯变是工程数学中常用的一种积分变换。它是为简化计算而建立的实变量函数和复变量函数间的一种函数变换。对一个实变量函数作拉普拉斯变换,并在复数域中作各种运算,再将运算结果作拉普拉斯反变换来求得实数域中的相应结果,往往比直接在实数域中求出同样的结果在计算上容易得多。拉普拉斯变换的这种运算步骤对于求解线性微分方程尤为有效,它可把微分方程化为容易求解的代数方程来处理,从而使计算简化。在经典控制理论中,对控制系统的分析和综合,都是建立在拉普拉斯变换的基础上的。引入拉普拉斯变换的一个主要优点,是可采用传递函数代替微分方程来描述系统的特性。这就为采用直观和简便的图解方法来确定控制系统的整个特性(见信号流程图、动态结构图)、分析控制系统的运动过程(见奈奎斯特稳定判据、根轨迹法),以及综合控制系统的校正装置(见控制系统校正方法)提供了可能性。拉普拉斯变换在工程学上的应用:应用拉普拉斯变换解常变量齐次微分方程,可以将微分方程化为代数方程,使问题得以解决。在工程学上,拉普拉斯变换的重大意义在于:将一个信号从时域上,转换为复频域(s域)上来表示;在线性系统,控制自动化上都有广泛的应用。

拉普拉斯变换是以法国数学家拉普拉斯命名的一种变换方法,主要是针对连续信号的分析。

傅里叶变换虽然好用,而且物理意义明确,但有一个最大的问题是其存在的条件比较苛刻,比如时域内绝对可积的信号才可能存在傅里叶变换。拉普拉斯变换可以说是推广了这以概念。在自然界,指数信号exp(-x)是衰减最快的信号之一,对信号乘上指数信号之后,很容易满足绝对可积的条件。因此将原始信号乘上指数信号之后一般都能满足傅里叶变换的条件,这种变换就是拉普拉斯变换。这种变换能将微分方程转化为代数方程,在18世纪计算机还远未发明的时候,意义非常重大。

从上面的分析可以看出,傅里叶变换可以看做是拉普拉斯的一种特殊形式,即所乘的指数信号为exp(0)。也即是说拉普拉斯变换是傅里叶变换的推广,是一种更普遍的表达形式。在进行信号与系统的分析过程中,可以先得到拉普拉斯变换这种更普遍的结果,然后再得到傅里叶变换这种特殊的结果。这种由普遍到特殊的解决办法,已经证明在连续信号与系统的分析中能够带来很大的方便。

Z-Transform

The Z-transform converts a discrete-time signal into a complex frequency domain representation. It can be considered as a discrete-time equivalent of the Laplace transform.

Z变换可以说是针对离散信号和系统的拉普拉斯变换,由此我们就很容易理解Z变换的重要性,也很容易理解Z变换和傅里叶变换之间的关系。Z变换中的Z平面与拉普拉斯中的S平面存在映射的关系,z=exp(Ts)。在Z变换中,单位圆上的结果即对应离散时间傅里叶变换的结果。

$Z$-transform (here Unilateral) of a sequence $\{a_k\}_{k=0}^\infty$ is defined as $$ X(z) \equiv Z[\{a_k\}_{k=0}^\infty](z) = \sum_{k=0}^\infty \frac{a_k}{z^k} $$ $z \in \mathbb{C}$.

Bilateral Z-transform: $$ X(z) \equiv Z[\{a_k\}_{k=-\infty}^\infty](z) = \sum_{k=-\infty}^\infty \frac{a_k}{z^k} $$ which is less used. So in the following, Z-transform is unilateral by default.

WCF note. A sequence is a function: $f: \mathbb{Z}^+ \rightarrow \mathbb{R}$, or $\mathbb{C}$, which could be represented as $f_n$, $f(n)$, or $\{a_n\}$.

Inverse $Z$-transform of a sequence is not unique unless its region of convergence is specified (Zwillinger 1996). If the Z-transform $F(z)$ of a function is known analytically, the inverse Z-transform can be computed using the contour integral $$ \{a_n\}_{n=0}^\infty = Z^{-1}[F(z)](n) = \frac{1}{2\pi i}\oint_\gamma F(z)z^{n-1}dz$$ where $\gamma$ is a closed contour surrounding the origin of the complex plane in the domain of analyticity of $F(z)$.

Examples

Unit impulse function: $$ \delta[n] = \begin{cases} 1, & n = 0 \\ 0, & n \ne 0 \end{cases} $$ Unit step (Heaviside) function: $$ u[n] = \begin{cases} 1, & n \ge 0 \\ 0, & n < 0 \end{cases} $$ Unit step function is the CDF of delta function.

$$ Z[\delta[n]](z) = \frac{1}{z^0} = 1 $$ $$ Z[\delta[n-n_0]](z) = \frac{1}{z^{n_0}} $$ $$ Z[u[n]](z) = \sum_n \frac{1}{z^n}$$ $$ \forall |z|\gt 1, \quad Z[u[n]](z) = \frac{1}{1-z^{-n}} $$

Relationship to Fourier Transform

For values of $z$ in the region $|z|=1$, known as the unit circle, $z=ej\omega$: $$ \sum_{n=-\infty}^{\infty} x[n]\ z^{-n} = \sum_{n=-\infty}^{\infty} x[n]\ e^{-j\omega n} $$ known as the discrete-time Fourier transform (DTFT) of the x[n] sequence.

Euler-Lagrange Differential Equation

FAQ

Solve equation 1

Stackexchange. $$ \frac{y}{x} = \frac{5y−2x}{2x−2y} $$ Let $y=xt$, then: $t=\frac{5xt−2x}{2x−2xt}=\frac{5t−2}{2−2t}$. You can solve for $t$, and can go from here.

Lagrangian and Eulerian specification of the flow field

Wikipedia.