All notes
Calculus

# Basics

## Definitions of limit

### ($\epsilon$, $\delta$)-definition of limit

• A formalization of the notion of limit.
• Whenever a point $x$ is within $\delta$ units of $c$, $f(x)$ is within $\epsilon$ units of $L$.
• $$\lim_{x\rightarrow c} f(x) = L \Longleftrightarrow (\forall \epsilon >0)(\exists \delta >0)(\forall x \in \mathbf{D})(0<|x-c|<\delta \Rightarrow |f(x)-\mathbf{L}| < \epsilon)$$

### Infinitesimal definition

$$\lim_{x\to a} f(x) = L$$ if and only if whenever the difference $x−a$ is infinitesimal, the difference $f(x)−L$ is infinitesimal, as well, or in formulas: if $st(x) = a$ then $st(f(x)) = L$

infinitesimal
it is smaller than any real number, yet it is greater than zero.
Hyperreal
Hyperreal $*\mathbf{R}$ is the extended $\mathbf{R}$ plus infinitesimal and infinities, without changing any of the elementary axioms of algebra.
Start part function
In non-standard analysis, the standard part function is a function from the limited (finite) hyperreal numbers to the real numbers.
non-standard calculus
is the modern application of infinitesimals, in the sense of non-standard analysis, to differential and integral calculus.

## Continuity

• A function $f$ is continuous at $c$ if it is both defined at $c$ and its value at $c$ equals the limit of $f$ as $x \rightarrow c$.

## Derivative

The derivative of the function $y = f(x)$: $$\frac{dy}{dx} := \lim_{\Delta x \rightarrow 0} \frac{\Delta y}{\Delta x} = \lim_{\Delta x \rightarrow 0} \frac{f(x+\Delta x) - f(x)}{\Delta x}$$

It is the process turning secant lines into the tangent lines. .

### Power rule

$$\frac{d x^p}{dx} := \lim_{\Delta x \rightarrow 0} \frac{(x+\Delta x)^p - x^p}{\Delta x} = \lim_{\Delta x \rightarrow 0} x^{p-1} p + \binom{p}{2} x^{p-2} \Delta x + \binom{p}{3} x^{p-3} \Delta x^2 + \ldots$$ $$= x^{p-1} p$$

### Exponential rule

$${\frac {d}{dx}}a^x = a^x {\ln a}$$ $${\frac {d}{dx}}e^{ax} = ae^{ax}$$

### Natural logrithm, $e$

$e$ is defined as by limit: $$e :=\quad \lim _{n\to \infty }\left(1+{\frac {1}{n}}\right)^{n}$$

or by the infinite series: $$e := \sum_{k=0}^\infty 1/(k!)$$ as first published by Newton (1669; reprinted in Whiteside 1968, p. 225).

And $e$ is the unique number with the property that the area of the region bounded by the hyperbola y=1/x, the x-axis, and the vertical lines x=1 and x=e is 1. In other words, $$int_1^e(dx)/x=ln e=1$$

Now we find the derivative of $\ln x$ using the formal definition of the derivative: $${\frac {d}{dx}}\ln(x)=\quad \lim _{\Delta x\to 0}{\frac {\ln(x+\Delta x)-\ln(x)}{\Delta x}}=$$ $$\quad \lim _{\Delta x\to 0}{\frac {1}{\Delta x}}\ln \left({\frac {x+\Delta x}{x}}\right)=\quad \lim _{\Delta x\to 0}\ln \left({\frac {x+\Delta x}{x}}\right)^{\frac {1}{\Delta x}}$$

Let $n={\frac {x}{\Delta x}}$. Note that as $n$ approaches $\infty$ , $\Delta x$ approaches 0. So we can redefine our limit as: $$\quad \lim _{n\to \infty }\ln \left(1+{\frac {1}{n}}\right)^{\frac {n}{x}}={\frac {1}{x}}\ln \left(\lim _{n\to \infty }\left(1+{\frac {1}{n}}\right)^{n}\right)={\frac {1}{x}}\ln(e)={\frac {1}{x}}$$

### Logarithm Rule

wikibooks.org: derivatives of exponential and logarithm functions. $${\frac {d}{dx}}\ln(x)=\quad \lim _{\Delta x\to 0}{\frac {\ln(x+\Delta x)-\ln(x)}{\Delta x}} =$$ $$\quad \lim _{\Delta x\to 0}{\frac {1}{\Delta x}}\ln \left({\frac {x+\Delta x}{x}}\right)$$

Let $n={\frac {x}{\Delta x}}$. Note that as $n$ approaches $\infty$, $\Delta x$ approaches 0. So we can redefine our limit as:

$$\quad \lim _{n\to \infty }\ln \left(1+{\frac {1}{n}}\right)^{\frac {n}{x}}={\frac {1}{x}}\ln \left(\lim _{n\to \infty }\left(1+{\frac {1}{n}}\right)^{n}\right)={\frac {1}{x}}\ln(e)={\frac {1}{x}}$$

$${\frac {d}{dx}}\log_b(x) = \frac{1}{x} \frac{1}{\ln b}$$

### Properties of Derivatives

$$\frac{d}{dx} cf(x) = c \frac{df}{dx}$$ $$[cf(x)]' = cf'(x)$$

Sum and Difference Rules $$\frac{d}{dx} [f(x) \pm g(x)] = \frac{df}{dx} \pm \frac{dg}{dx}$$ $$[f(x) \pm g(x)]' = f'(x) \pm g'(x)$$

## Other

$$\frac{d}{dx} \left( \frac{1}{1-x} \right) = \frac{-1}{(1-x)^2} \cdot \frac{d}{dx} (1-x) = \frac{1}{(1-x)^2}$$

## Taylor Series

Taylor Series
A series expansion of a function about a point.
Maclaurin Series
Taylor series with $a=0$.

Taylor Series: $$f(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f^{(3)}(a)}{3!}(x-a)^3 + \ldots$$ $$f(x) = \sum_{i=0}^{\infty} \frac{f^{i}(a)}{i!}(x-a)^i$$

### Taylor's theorem

Any function satisfying certain conditions can be expressed as a Taylor Series.

# Lagrange Multiplier

• It is used to find the extrema of a multivariate function $f(x_1,x_2,...,x_n)$ subject to the constraint $g(x_1,x_2,...,x_n)=0$, where $f$ and $g$ are functions with continuous first partial derivatives on the open set containing the curve $g(x_1,x_2,...,x_n)=0$, and $\nabla g \neq 0$ at any point on the curve (where $\nabla$ is the gradient).
• For an extremum of $f$ to exist on $g$, the gradient of $f$ must line up with the gradient of $g$.
• If the two gradients are in the same direction, then one is a multiple ($-\lambda$) of the other, so $$\nabla f = -\lambda \nabla g$$ E.g. $$\frac{\partial f}{\partial x_k} + \lambda \frac{\partial g}{\partial x_k} = 0$$ $\forall k=1, \ldots, n$, where the constant $\lambda$ is called the Lagrange multiplier.
• The extremum is then found by solving the n+1 equations in n+1 unknowns, which is done without inverting $g$, which is why Lagrange multipliers can be so useful.

Lamar. Method of Lagrange Multipliers

• Solve the following system of equations (4 equations here for 4 unknowns $x,y,z,\lambda$): $$\nabla f(x,y,z) = \lambda \nabla g(x,y,z)$$ $$g(x,y,z) = 0$$
• Plug in all solutions, $(x,y,z)$ , from the first step into $f(x,y,z)$ and identify the minimum and maximum values, provided they exist.
Exams. See the ref before for answer.
• Find the dimensions of the box with largest volume if the total surface area is 64 $\text{cm}^2$.
• Find the maximum and minimum of $f(x,y)=5x-3y$ subject to the constraint $x^2+y^2=136$.

#### A good explanation

Wikipedia. See good diagrams in the reference.

• Suppose we walk along the contour line with $g = 0$. We are interested in finding points where $f$ does not change as we walk, since these points might be maxima. There are two ways this could happen:
• We could be following a contour line of $f$, since by definition $f$ does not change as we walk along its contour lines. This would mean that the contour lines of $f$ and $g$ are parallel here.
• The second possibility is that we have reached a "level" part of $f$, meaning that $f$ does not change in any direction.
• To check the first possibility, notice that since the gradient of a function is perpendicular to the contour lines, the contour lines of $f$ and $g$ are parallel if and only if the gradients of $f$ and $g$ are parallel: $\nabla_{x,y} f = - \lambda \nabla_{x,y} g$. WcfNote: the minus sign here is just for simplicity, and it's irrevalent.
• This method also solves the second possibility: if $f$ is level, then its gradient is zero, and setting $\lambda$ = 0 is a solution regardless of $g$.
• To incorporate these conditions into one equation, we introduce an auxiliary function $$\Lambda(x,y,\lambda) = f(x,y) + \lambda \cdot g(x,y)$$ and solve $$\nabla_{x,y,\lambda} \Lambda(x , y, \lambda)=0$$ Note that $\nabla_{\lambda} \Lambda(x , y, \lambda)=0$ implies $g(x, y) = 0$.

# Integral Transform

## Z-Transform

The Z-transform converts a discrete-time signal into a complex frequency domain representation. It can be considered as a discrete-time equivalent of the Laplace transform.

Z变换可以说是针对离散信号和系统的拉普拉斯变换，由此我们就很容易理解Z变换的重要性，也很容易理解Z变换和傅里叶变换之间的关系。Z变换中的Z平面与拉普拉斯中的S平面存在映射的关系，z=exp(Ts)。在Z变换中，单位圆上的结果即对应离散时间傅里叶变换的结果。

$Z$-transform (here Unilateral) of a sequence $\{a_k\}_{k=0}^\infty$ is defined as $$X(z) \equiv Z[\{a_k\}_{k=0}^\infty](z) = \sum_{k=0}^\infty \frac{a_k}{z^k}$$ $z \in \mathbb{C}$.

Bilateral Z-transform: $$X(z) \equiv Z[\{a_k\}_{k=-\infty}^\infty](z) = \sum_{k=-\infty}^\infty \frac{a_k}{z^k}$$ which is less used. So in the following, Z-transform is unilateral by default.

WCF note. A sequence is a function: $f: \mathbb{Z}^+ \rightarrow \mathbb{R}$, or $\mathbb{C}$, which could be represented as $f_n$, $f(n)$, or $\{a_n\}$.

Inverse $Z$-transform of a sequence is not unique unless its region of convergence is specified (Zwillinger 1996). If the Z-transform $F(z)$ of a function is known analytically, the inverse Z-transform can be computed using the contour integral $$\{a_n\}_{n=0}^\infty = Z^{-1}[F(z)](n) = \frac{1}{2\pi i}\oint_\gamma F(z)z^{n-1}dz$$ where $\gamma$ is a closed contour surrounding the origin of the complex plane in the domain of analyticity of $F(z)$.

### Examples

Unit impulse function: $$\delta[n] = \begin{cases} 1, & n = 0 \\ 0, & n \ne 0 \end{cases}$$ Unit step (Heaviside) function: $$u[n] = \begin{cases} 1, & n \ge 0 \\ 0, & n < 0 \end{cases}$$ Unit step function is the CDF of delta function.

$$Z[\delta[n]](z) = \frac{1}{z^0} = 1$$ $$Z[\delta[n-n_0]](z) = \frac{1}{z^{n_0}}$$ $$Z[u[n]](z) = \sum_n \frac{1}{z^n}$$ $$\forall |z|\gt 1, \quad Z[u[n]](z) = \frac{1}{1-z^{-n}}$$

### Relationship to Fourier Transform

For values of $z$ in the region $|z|=1$, known as the unit circle, $z=ej\omega$: $$\sum_{n=-\infty}^{\infty} x[n]\ z^{-n} = \sum_{n=-\infty}^{\infty} x[n]\ e^{-j\omega n}$$ known as the discrete-time Fourier transform (DTFT) of the x[n] sequence.

# Euler-Lagrange Differential Equation

• It is the fundamental equation of Calculaus of Variations.
• If $J$ is defined by $$J = \int f(t,y,\dot{y})dt,$$ then $J$ has a stationary value when the E-L equation $$\frac{\partial f}{\partial y} - \frac{d}{dt}(\frac{\partial f}{\partial \dot{y}}) = 0$$ is satisfied.
• In many phisical problems, $f_x$ turns out to be 0, and the E-L equation reduces to the Beltrami identity: $$f-y_x \frac{\partial f}{\partial y_x} = C.$$

# FAQ

#### Solve equation 1

Stackexchange. $$\frac{y}{x} = \frac{5y−2x}{2x−2y}$$ Let $y=xt$, then: $t=\frac{5xt−2x}{2x−2xt}=\frac{5t−2}{2−2t}$. You can solve for $t$, and can go from here.

# Lagrangian and Eulerian specification of the flow field

• The Lagrangian specification of the field is a way of looking at fluid motion where the observer follows an individual fluid parcel as it moves through space and time. Plotting the position of an individual parcel through time gives the pathline of the parcel.
• The Eulerian specification of the flow field is a way of looking at fluid motion that focuses on specific locations in the space through which the fluid flows as time passes.
• In the Eulerian specification of a field, it is represented as a function of position $x$ and time $t$. For example, the flow velocity is represented by a function $\mathbf{u}\left(\mathbf{x}(t), t\right)$
• In the Lagrangian description, the flow is described by a function $\mathbf{U}\left(\mathbf{x}_0,t\right)$ giving the position of the parcel labeled $x_0$ at time $t$.
• The two specifications are related as follows: $$\mathbf{u}\left(\mathbf{U}(\mathbf{x}_0,t),t \right) = \frac{\partial \mathbf{U}}{\partial t}\left(\mathbf{x}_0,t \right)$$ because both sides describe the velocity of the parcel labeled $x_0$ at time $t$.
• Within a chosen coordinate system, $x_0$ and $x$ are referred to as the Lagrangian coordinates and Eulerian coordinates of the flow.
• Substantial derivative: Suppose we have a flow field $u$, and we are also given a generic field with Eulerian specification $F(x,t)$. Now one might ask about the total rate of change of $F$ experienced by a specific flow parcel. This can be computed as $$\frac{\mathrm{D}\mathbf{F}}{\mathrm{D}t} = \frac{\partial \mathbf{F}}{\partial t} + (\mathbf{u}\cdot \nabla)\mathbf{F}$$ This tells us that the total rate of change of the function $F$ as the fluid parcels moves through a flow field described by its Eulerian specification $v$ is equal to the sum of the local rate of change and the convective rate of change of $F$. This is a consequence of the chain rule since we are differentiating the function $F(U(x0,t),t)$ with respect to $t$.
• Conservation laws for a unit mass have a Lagrangian form, which together with mass conservation produce Eulerian conservation; on the contrary when fluid particle can exchange the quantity (like energy or momentum) only Eulerian conservation law exists, see Falkovich.