MIT OCW

18.02 Multivariable Calculus

Total Differentials

When we are considering a multivariable function, is there a way to hold changes of all components?
Well there's the total differential defined for $f(x,y,z)$ that is $$\mathrm{d}f=\frac{\partial f}{\partial x}\mathrm{d}x+\frac{\partial f}{\partial y}\mathrm{d}y+\frac{\partial f}{\partial z}\mathrm{d}z.$$

Notice I've been confusing the derivatives and differentials, but I'm now clearing the edge between them. In single variable situation, when apply "differential" to some function $f(x)$, we get actually another function $$\mathrm{d}f(x,\Delta x)\overset{\Delta}{=}f'(x)\Delta x.$$
We often write something like $$\mathrm{d}f=\boxed{}\mathrm{d}x$$ because according to definition $$\mathrm{d}(x,\Delta x)=\Delta x.$$

Chain Rule

If we have some multivariable function $f(x,y,z)$, where $x=x(t),y=y(t),z=z(t)$, we can get $$\frac{\mathrm{d}f}{\mathrm{d}t}=\frac{\partial f}{\partial x}\frac{\mathrm{d}x}{\mathrm{d}t}+\frac{\partial f}{\partial y}\frac{\mathrm{d}y}{\mathrm{d}t}+\frac{\partial f}{\partial z}\frac{\mathrm{d}z}{\mathrm{d}t}.$$

Validation for Product and Quotient Rule

Treat product of two functions $u=u(t),v=v(t)$ as a multivariable function $f(u,v)=uv$, and apply the chain rule $$\begin{aligned}\frac{\mathrm{d}f}{\mathrm{d}t}&=\frac{\partial f}{\partial u}\frac{\mathrm{d}u}{\mathrm{d}t}+\frac{\partial f}{\partial v}\frac{\mathrm{d}v}{\mathrm{d}t}\\&=v\frac{\mathrm{d}u}{\mathrm{d}t}+u\frac{\mathrm{d}v}{\mathrm{d}t}.\end{aligned}$$
The quotient rule can be validated similarly, omitted.

Chain Rule for Several Variables

Given a function $f(x,y)$ where $x=x(u,v),y=y(u,v)$, how to get $\frac{\partial f}{\partial u}$ and $\frac{\partial f}{\partial v}$ without plugging $x=x(u,v)$ and $y=y(u,v)$ in?
Let's calculate the total differential of $f$, that is $$\begin{aligned}\mathrm{d}f&=\frac{\partial f}{\partial x}\mathrm{d}x+\frac{\partial f}{\partial y}\mathrm{d}y\\&=\frac{\partial f}{\partial x}\left(\frac{\partial x}{\partial u}\mathrm{d}u+\frac{\partial x}{\partial v}\mathrm{d}v\right)+\frac{\partial f}{\partial y}\left(\frac{\partial y}{\partial u}\mathrm{d}u+\frac{\partial y}{\partial v}\mathrm{d}v\right)\\&=\left(\frac{\partial f}{\partial x}\frac{\partial x}{\partial u}+\frac{\partial f}{\partial y}\frac{\partial y}{\partial u}\right)\mathrm{d}u+\left(\frac{\partial f}{\partial x}\frac{\partial x}{\partial v}+\frac{\partial f}{\partial y}\frac{\partial y}{\partial v}\right)\mathrm{d}v.\end{aligned}$$
And notice that $$\mathrm{d}f=\frac{\partial f}{\partial u}\mathrm{d}u+\frac{\partial f}{\partial v}\mathrm{d}v,$$ therefore we get $$\left\{\begin{aligned}\frac{\partial f}{\partial u}&=\frac{\partial f}{\partial x}\frac{\partial x}{\partial u}+\frac{\partial f}{\partial y}\frac{\partial y}{\partial u}\\\frac{\partial f}{\partial v}&=\frac{\partial f}{\partial x}\frac{\partial x}{\partial v}+\frac{\partial f}{\partial y}\frac{\partial y}{\partial v}.\end{aligned}\right.$$

MIT OCW

18.02 Multivariable Calculus

Basics of Multivariable Functions

Map

At a higher point of view, a single variable function maps some number to a number, and a multivariable function maps some $n$-tuple to a number. The essence of a function doesn't change.

Domain

Like a single variable function, a multivariable function have its domain, like $$f(x,y)=x^2+y^2$$ can be defined all the time, and $$f(x,y)=\sqrt{y}$$ is only defined when $y\geq 0$.

Graph

It's difficult to plot a multivariable function accurately, but the main idea doesn't change.
e.g. Plot the graph of $f(x,y)=-y$.
Consider the $yz$ plane, it's just a line through origin. And now we move the value of $x$, and it doesn't depend on the value $x$, so it will be a plane.
z=-y.png

e.g. Plot the graph of $f(x,y)=1-x^2-y^2$.
Consider the $yz$ plane, where $x=0$, it will be a parabola with the quation $z=1-y^2$. Similarly, the part on $xz$ plane is still a parabola with the quation $z=1-x^2$.
But if we consider the graph on the $xy$ plane, where $z=0$, we'll get a unit circle that is $x^2+y^2=1$.
1-x2-y2.png

Contour Plot

The process of a traditional plot is hard and not that easy to understand. Take the last example, we can draw a contour plot like the following
contour.png
which indicated where the function achieves the same value on the $xy$ plane.
We can feel the change of the function by observing the gap between curves and know how the function changes along some direction.

Partial Derivatives

We care about the change rate of a multivariable function, but it has several variables. What we do is just convert the multivariable function into a single variable function, that is treat other variables as constants.
Given a function $f(x,y)$, the partial derivative at point $(x_0,y_0)$ of $f$ with respect to $x$ is $$\frac{\partial f}{\partial x}(x_0,y_0)=\lim_{\Delta x\to 0}\frac{f(x_0+\Delta x, y_0)-f(x_0, y_0)}{\Delta x},$$ and generally the partial derivative of $f$ with respect to $x$ is a multivariable function that is $$f_x=\frac{\partial f}{\partial x}=\lim_{\Delta x\to 0}\frac{f(x+\Delta x, y)-f(x, y)}{\Delta x}.$$

Approximation

Like the linear approximation in single variable function, a multivariable function also has approximation. That is
$$\Delta f(x,y)\approx f_x\Delta x+f_y\Delta y.$$
Why is it correct?
Thinking about the following assumption $$\left\{\begin{aligned}f_x(x_0,y_0)=a\\f_y(x_0,y_0)=b\end{aligned}\right.,$$
and we have two tangent lines $$l_1:\left\{\begin{aligned}z&=z_0+a(x-x_0)\\y&=y_0\end{aligned}\right.,l_2:\left\{\begin{aligned}z&=z_0+b(y-y_0)\\x&=x_0\end{aligned}\right..$$
And these two lines can determine a tangent plane $z=z_0+a(x-x_0)+b(y-y_0)$.

Maxima / Minima

At local maxima or minima, we have $f_x(x_0,y_0)=0$ and $f_y(x_0,y_0)=0$, which means the point has a horizontal tangent plane.

Critical Points

Point $(x,y)$ is a critical point of $f$ if $f_x(x_0,y_0)=0$ and $f_y(x_0,y_0)=0$.
e.g. Find critical points for $f(x,y)=x^2-2xy+3y^2+2x-2y$.
Solution According to definition, we get $$\left\{\begin{aligned}f_x&=2x-2y+2\\f_y&=-2x+6y-2=0\end{aligned}\right.,$$ and we can solve the critical point that is $(-1,0)$.
But what's the minima?
Notice that $$f(x,y)=x^2-2xy+3y^2+2x-2y=(x-y+1)+2y^2-1\geq -1,$$ and we plug $(-1,0)$ into it, it's exactly $-1$, so we get local and global minima at $(-1,0)$.

Saddle Points

Focus on the condition $f_x(x_0,y_0)=0$ and $f_y(x_0,y_0)=0$, it's neccessary but not sufficient in terms of maxima or minima. Because we could have some examples that $f_x(x_0,y_0)=0$ and $f_y(x_0,y_0)=0$ where the function doesn't have local minima or maxima at $(x_0,y_0)$.
For example, the function $f(x,y)=x^2-y^2$.
x2-y2.png
It's easy to validate that $f_x(0,0)=0,f_y(0,0)=0$, anyhow, it doesn't achieve maxima or minima at the point, these points are called saddle points.

Application

Least-square Interpolation

When we are given a lot of discrete data (often seen in scientific experiments), we want to find some line to approximate these data, and enable us to find the relation between variables and predict their value.
Considering the data set $(x_1,y_1),(x_2,y_2),\cdots,(x_n,y_n)$ and the target line $y=ax+b$, how to optimize the line in order that the line become best for discrete data? Actually, it's a minima problem.
As a convention, the indicator we use here is the offset squared, for each data $(x_i,y_i)$ that is $(ax_i+b-y_i)^2$, so the function we'd like to analyze is $$D(a,b)=\sum_1^n (ax_i+b-y_i)^2,$$ whose minima we want to find.
We take the partial derivative with respect to $a,b$
$$\frac{\partial D}{\partial a}=\sum_1^n [2(ax_i+b-y_i)x_i],\frac{\partial D}{\partial b}=\sum_1^n [2(ax_i+b-y_i)],$$ and to get the critical point, we solve the system $$\left\{\begin{aligned}&\frac{\partial D}{\partial a}=0\\&\frac{\partial D}{\partial b}=0\end{aligned}\right.,$$ namely $$\left\{\begin{aligned}&\left(\sum_1^nx_i^2\right)a+\left(\sum_1^nx_i\right)b=\sum_1^nx_iy_i\\&\left(\sum_1^nx_i\right)a+nb=\sum_1^ny_i\end{aligned}\right..$$

Validation by Second Derivatives

Special Cases

Completing Squares

Consider the following function behavior at origin $$f(x,y)=ax^2+bxy+cy^2,$$ and we try to complete the square here to judge whether the function can achieve maxima or minima at origin, namely $$\begin{aligned}f(x,y)&=a\left(x^2+\frac{b}{a}xy\right)+cy^2\\&=a\left(x+\frac{b}{2a}y\right)^2-\frac{b^2}{4a}y^2+cy^2\\&=a\left(x+\frac{b}{2a}y\right)^2+\frac{4ac-b^2}{4a}y^2\\&=\frac1{4a}\left[4a^2\left(x+\frac{b}{2a}y\right)^2+(4ac-b^2)y^2\right].\end{aligned}$$
It's obvious that the origin is a critical point because $$\frac{\partial f}{\partial x}=2ax+by,\frac{\partial f}{\partial y}=2cy+bx.$$ And plug the origin into these, easy to get they are $0$, the problem here is can $f(0,0)$ be the true local maxima or minima?
Observe the completed part, the signs before two squared terms are interesting, $4a^2$ is always positive, and $4ac-b^2$ is not determined.
If $4ac-b^2>0$, then the parts in bracket are two non-negative terms, and have to be non-negative. And $f(0,0)=0$, then it has local maxima or minima (depending on the sign of $\frac1{4a}$, namely $a$).
If $4ac-b^2=0$, the function will depends on only one variable $x$, and the behavior here cannot be concluded. In this special case, it will be local maxima or minima, where any point $(0,t)$ will achive.
If $4ac-b^2<0$, then the parts in bracket are one non-negative, the other non-positive, so it's possible to get either positive or negative value. So $f(0,0)$ cannot be maxima or minima, it's a saddle point.

Homogenous Equation

We find that quadratic discriminant $b^2-4ac$ occur in the analysis above, is that a coincidence?
Still the example above, notice that each term is quadratic, so $$f(x,y)=y^2\left(a\left(\frac{x}{y}\right)^2+b\left(\frac{x}{y}\right)+c\right),$$ and near the origin, $\frac{x}{y}$ can be any number.
If the equation $at^2+bt+c=0$ has two roots, namely, $b^2-4ac>0$, it means that the function can achieve two sides of $0$, and it keeps $0$ on some direction ($\frac{x}{y}$ indicates the direction when approaching the origin).
quad-root-up.pngquad-root-down.png
If the equation $at^2+bt+c=0$ has only one root, namely, $b^2-4ac=0$, actually it can't be concluded, anyhow, it indicates that on some direction, the function keeps its value at origin.
If the equation $at^2+bt+c=0$ has no roots, namely, $b^2-4ac<0$, it means that $f(0,0)=0$ will be exactly local maxima or minima (depending on the sign of $a$), because near the origin, the function value can't be zero.
quad-noroot.png

Second Derivative Test

According to multivariable quadratic Taylor's formula $$\Delta f\approx f_x(x-x_0)+f_y(y-y_0)+\frac12f_{xx}(x-x_0)^2+f_{xy}(x-x_0)(y-y_0)+\frac12f_{yy}(y-y_0)^2,$$
we can have a general test on other functions.
To test a critical point $(x_0,y_0)$ of $f$, let $$A=f_{xx}(x_0,y_0),B=f_{xy}(x_0,y_0),C=f_{yy}(x_0,y_0),$$

  • if $AC-B^2>0$
    • if $A>0$, we get local minima at $(x_0,y_0)$
    • if $A<0$, we get local maxima at $(x_0,y_0)$
  • if $AC-B^2<0$, then it's a saddle point
  • if $AC-B^2=0$, no conclusion

MIT OCW

18.02 Multivariable Calculus

Motion

Recall the cycloid, we use a parametric equation to describe some point $P(x(t),y(t),z(t))$. The vector $\mathbf{r}(t)=\mathbf{OP}=(x(t),y(t),z(t))$ is called position vector, and we can learn about more details when analysing the vector.
Take a cycloid where $t=\theta$ as an example that is $$\mathbf{r}(t)=(t-\sin(t),1-\cos(t)),$$ though it hasn't the third component, but it doesn't lose generality.

Velocity

We care about not only the rate at some point but also the direction, in multivariable calculus, a vector can be diffentiated. To get the velocity, we just differentiate the position vector $$\mathbf{v}=\frac{\mathrm{d}\mathbf{r}}{\mathrm{d}t}.$$ In this cycloid case, $$\mathbf{v}=\frac{\mathrm{d}\mathbf{r}}{\mathrm{d}t}=\left(\frac{\mathrm{d}x}{\mathrm{d}t},\frac{\mathrm{d}y}{\mathrm{d}t}\right)=(1-\cos(t),\sin(t)).$$

Speed

In some applications, we only care about the rate. The magnitude of velocity becomes speed, in this example that is $$|\mathbf{v}|=\sqrt{(1-\cos(t))^2+\sin^2(t)}=\sqrt{2-2\cos(t)}.$$

Acceleration

Image you are driving, and you take a tight turn without change of the speed, in the view of single variable calculus, you do not have acceleration. Anyhow, in multivariable calculus, your change of direction is also taken into consideration. Similarly defined like velocity, we have the acceleration $$\mathbf{a}=\frac{\mathrm{d}\mathbf{v}}{\mathrm{d}t}.$$
In this cycloid case, $$\mathbf{a}=\frac{\mathrm{d}\mathbf{v}}{\mathrm{d}t}=(\sin(t),\cos(t)).$$

Arc Length

If we add velocity continuously, we can get a vector from the start point to the end point. How can we get the arc length during the process? A good idea is to add speed continuously, because speed doesn't have direction. So we have $$\frac{\mathrm{d}s}{\mathrm{d}t}=|\mathbf{v}|.$$
e.g. Length of an arch of cycloid is $\int_0^{2\pi}\sqrt{2-2\cos(t)}\mathrm{d}t$

Trajectory Unit Tangent Vector

Trajectory unit tangent vector is defined as $\hat{\mathbf{T}}=\frac{\mathbf{v}}{|\mathbf{v}|}$.
And we notice that $$\mathbf{v}=\frac{\mathrm{d}\mathbf{r}}{\mathrm{d}t}=\mathbf{v}=\frac{\mathrm{d}\mathbf{r}}{\mathrm{d}s}\frac{\mathrm{d}s}{\mathrm{d}t},$$ where $\frac{\mathrm{d}s}{\mathrm{d}t}$ is actually $|\mathbf{v}|$, so we have $$\hat{\mathbf{T}}=\frac{\mathrm{d}\mathbf{r}}{\mathrm{d}s}.$$

Second Law of Kepler

[TO BE CONTINUED]

MIT OCW

18.02 Multivariable Calculus

Equations of Lines

We have known that a line can be treated as the intersection of two planes, so it's okay to describe a line by two equations, but not that natural.
Here's another way, a line can be treated as the trajectory of a moving point.

e.g. Line through $Q_0(-1,2,2),Q_1(1,3,-1)$
Solution Select a point $Q(x,y,z)$ on the plane, and we should have $\mathbf{Q_0Q}=t\mathbf{Q_0Q_1}=(2,1,-3)$, for some $t$. Let's consider $t$ as a parameter and expand the formula that is $$\begin{bmatrix}x+1\\y-2\\z-2\end{bmatrix}=t\begin{bmatrix}2\\1\\-3\end{bmatrix}.$$
We can write $x,y,z$ as functions with respect to $t$ as the following $$\left\{\begin{aligned}x(t)&=&-1&+2t\\y(t)&=&2&+t\\z(t)&=&2&-3t\end{aligned}\right..$$

Intersection with a Plane

e.g. What's $Q_0(-1,2,2)$ and $Q_1(1,3,-1)$ like relative to $x+2y+4z=7$?
Same side / Opposite sides / One is on the plane.
Solution The plane $x+2y+4z=7$ is actually dividing the whole space into two regions where one fulfills $x+2y+4z>7$ and the other $x+2y+4z<7$. We just plugin these two points into the plane equation and check the equality.
For $Q_0$, $-1+2\times 2+4\times 2=11>7$; and for $Q_1$, $1+2\times 3+4\times(-1)=3<7$, so neither of them is on the plane and they are on the opposite sides.
To get the intersection, we just plug the parametric equation of the line into the plane equation that is $$x(t)+2y(t)+4z(t)=7,$$
after solving, we can get $t=\frac12$, and indicating the intersection is $(0,\frac25,\frac12)$.

Parametric Equations of Curves

Cycloid

The whole process is just a trick in geometry, the parametric equation for a cycloid with radius $a$ is
$$ \left\{ \begin{aligned} x(\theta)&=a(\theta-\sin(\theta))\\ y(\theta)&=a(1-\cos(\theta)) \end{aligned} \right.. $$
Let's observe the edge like the place near $\theta=0,2\pi,\cdots$. Without loss of generality, take $\theta=0$ as an example.
According to Taylor's theorem, $\sin(\theta)\sim\theta-\frac{\theta^3}{3!}$ and $\cos\theta\sim1-\frac{\theta^2}{2!}$, consider the slope near $\theta=0$ that is $\frac{y(t)}{x(t)}=\frac{1-\cos(\theta)}{\theta-\sin(\theta)}~\frac{\frac{\theta^2}{2!}}{\frac{\theta^3}{3!}}=\frac{3}{\theta}\to\infty$, indicating that the behavior near $\theta=0$ is to toggle some motion, which is obvious to be understood when we imagine a wheel rolling on the ground.

MIT OCW

18.02 Multivariable Calculus

Equations of Planes

Expression

The equation of a plane should be in the form of $ax+by+cz=d$.
e.g. Plane through origin with normal vector $\mathbf{N}=(1,5,10)$.
Solution Select a point $P(x,y,z)$ on the plane and we have $\mathbf{OP}\cdot\mathbf{N}=0$, that is $x+5y+10z=0$.

e.g. Plane through $P_0(2,1,-1)$ with normal vector $\mathbf{N}=(1,5,10)$.
It's slightly different but what we will do is almost the same. Select a point $P(x,y,z)$ on the plane and we have $\mathbf{P_0P}\cdot\mathbf{N}=0$, that is $(x-2)+5(y-1)+10(z+1)=0$, simplified as $x+5y+10z=-3$.

Observe these two examples, their coefficients before $x,y,z$ are exactly the components of the normal vector $\mathbf{N}$, the only difference is just the constant. Here the constant can be understood as some offset to origin, since there is no offset when the constant is $0$, namely through the origin.

After we know the fact, we can solve the second example in an easier way. Because we've known the form in the second example is $x+5y+10z=d$, we just plug $P_0(2,1-1)$ into it and solve the value of $d$, finally we get the whole equation.

Position

If we know a vector and the equation of a plane, we can judge their relationship by the above trick.
e.g. Given a vector $\mathbf{v}=(1,2,-1)$ and a plane $x+y+3z=5$, what's their relationship? Parallel / Perpendicular / Neither.
Solution The normal vector of the plane is $\mathbf{N}=(1,1,3)$, and we find that $\mathbf{N}\cdot\mathbf{v}=0$. Caution! It DOES NOT mean $\mathbf{v}$ is perpendicular to the plane, instead, what we check is $\mathbf{v}$ and the normal vector $\mathbf{N}$ of the plane. Therefore, it oppositely suggests that the vector $\mathbf{v}$ is parallel to the plane.

Linear Systems

Think about a $3\times 3$ linear system, for example, $$\left\{\begin{alignedat}{4} x & {}+{} & & {}{} & z & {}={} & 1 \\ x & {}+{} & y & {} {} & & {}={} & 2 \\ x & {}+{} & 2y & {}+{} & 3z & {}={} & 3\end{alignedat}\right..$$
These three equations are all planes, and what's the meaning of a solution?
In geometry, we can only find only one solution when the first two planes intersect in a line, and the line intersects with the third plane with a point. To solve this, a good way is to use matrix, a.k.a. $\mathbf{A}\mathbf{X}=\mathbf{B}\Leftrightarrow\mathbf{X}=\mathbf{A}^{-1}\mathbf{B}$.
Anyhow, there exists some weird situtations and the method doesn't work. Generally, the solution of the system have four possibilities:

  • No solutions
  • One point (unique solution)
  • A line (infinite solutions)
  • A plane (infinite solutions)

If two planes are parallel, then there exists no solutions at all.
If three planes are all the same, then there exists infinite solutions, namely a plane.
If two planes intersect in a line, and the thrid plane is parallel to the line (not contained), then there exists no solutions.
If two planes intersect in a line, and the thrid plane is contain the line, then there exists infinite solutions, namely a line.

Invertible Matrices

What's above method going wrong in algebra? The formula $\mathbf{A}^{-1}=\frac{\operatorname{adj}(\mathbf{A})}{\det(\mathbf{A})}$ is definitely correct, but what will happen if $\det(\mathbf{A})=0$? We say that those matrices whose determinant $\det(\mathbf{A})\neq 0$ are invertible and they have inverse, the others are not invertible and don't have inverse.

Homogeneous Systems

A homogeneous system has the form $\mathbf{A}\mathbf{X}=0$,we can find that no matter what $\mathbf{A}$ is, obviously $\mathbf{X}=0$ is always a solution, which is called a trivial solution.
If $\det(\mathbf{A})\neq 0$, $\mathbf{X}=0$ is the unique solution.
If $\det(\mathbf{A})=0$, then $\det(\mathbf{N_1},\mathbf{N_2},\mathbf{N_3})=0$, which means their normal vectors are coplanar. Then there exists nontrivial solutions like $\mathbf{X}=\mathbf{N_1}\times\mathbf{N_2}$.

General Cases

For a system $\mathbf{A}\mathbf{X}=\mathbf{B}$

  • if $\det(\mathbf{A})\neq 0$, then it has unique solution
  • if $\det(\mathbf{A})=0$, then it has either no solutions or infinite solutions.

The more details will be found when trying to solve the system by elimination and substitution, but if we know $\det(\mathbf{A})=0$ and find a solution, then it has to have infinite solutions; and when we get something contradictory, then it turns out that the system has no solutions.