## Big Rudin: Chapter 4, Exercise 14

A week or so ago, a commenter requested that solutions to exercises 14 and 15 of Walter Rudin’s Real and Complex Analysis. The commenter failed to specify which chapter; but the comment was to a post for an exercise in chapter 4 of Baby Rudin, so I am guessing it’s chapter 4. It’s been an exhausting week at work, so I only have a solution to exercise 14. I’ll probably have exercise 14 ready later this weekend.

Underlying both exercises is the subject of orthogonal polynomials, which I encourage the reader to explore after solving the exercises; however, I will not discuss it here. We will make do with basic Hilbert space results and integral calculus.

The first result we need is on the orthogonal projection of an element $x$ onto a closed, convex set $E$ of a Hilbert space. Specifically, there is a unique element $s$ of $E$ which minimizes the distance between $x$ and $E$. If $E$ is a subspace, then $s$ also satisfies $x-s\perp M$. Moreover, this element $s$ can be computed using generalized Fourier series.

Lemma 1. Let $H$ be a Hilbert space, and let $\left\{e_{\alpha}:\alpha\in A\right\}$ be an orthonormal set, possibly uncountable, in $H$. Let $F$ be a finite subset of $A$, and define a subspace $M:=\text{span}\left\{e_{\alpha}:\alpha\in F\right\}$. Then

$\displaystyle\left\|x-s_{F}\right\|\leq\left\|x-s\right\|,\indent\forall s\in M$

where

$\displaystyle s_{F}:=\sum_{\alpha\in F}\langle{x,e_{\alpha}}\rangle e_{\alpha}$

Equality holds if and only if $s=s_{F}$.

We can drop the finiteness condition imposed on $F$ with a bit more work, but we will not need the full result for our purposes. I leave generalizing Lemma 1 as an exercise to the reader.

Proof. Since $F$ is finite, we can enumerate the set $\left\{e_{\alpha}:\alpha\in F\right\}$ by $\left\{e_{i}:1\leq i\leq n\right\}$, for some positive integer $n$. Observe that, for $1\leq j\leq n$,

$\begin{array}{lcl}\displaystyle\left\langle{x-s_{F},e_{j}}\right\rangle&=&\displaystyle\langle{x,e_{j}}\rangle-\langle{s_{F},e_{j}}\rangle\\[2 em]\displaystyle&=&\langle{x,e_{j}}\rangle-\sum_{i=1}^{n}\langle{x,e_{i}}\rangle\langle{e_{i},e_{j}}\rangle\\[2 em]\displaystyle&=&\langle{x,e_{j}}\rangle-\langle{x,e_{j}}\rangle\\[2 em]\displaystyle&=&0\end{array}$

It follows from the sesquilinearity of the inner product that $x-s_{F}\perp M$. Since $s_{F}-s\in M$, it follows from the Pythagorean theorem that

$\displaystyle\left\|x-s\right\|^{2}=\left\|(x-s_{F})+(s_{F}-s)\right\|^{2}=\left\|x-s_{F}\right\|^{2}+\left\|s_{F}-s\right\|^{2}$

It is evident that the inequality is strict unless $s=s_{F}$. $\Box$

Note that in the preceding lemma, $s_{F}$ is the unique solution to $\min_{s\in M}\left\|x-s\right\|^{2}$. But we can say more: $s_{F}$ is the unique element $s\in M$ with the property that $x-s\perp M$. To see this, observe that any such element $s$, by the Pythaogorean theorem, satisfies

$\displaystyle\left\|x-s\right\|^{2}=\left\|x-s_{F}\right\|^{2}+\left\|s_{F}-s\right\|^{2}$

From our earlier work, we conclude that $s=s_{F}$.

Consider the space $M$ of real polynomials with degree at most $2$. Define polynomials $p_{0}$, $p_{1}$, and $p_{2}$ by

$\displaystyle p_{0}(x):=1,\indent p_{1}(x):=x,\indent p_{2}(x):=x^{2}$

for all $x\in[-1,1]$. We use the Gram-Schmidt algorithm to construct an orthonormal basis $\left\{e_{0},e_{1},e_{2}\right\}$ out of the basis $\left\{p_{0},p_{1},p_{2}\right\}$ for the space $M$. We normalize $p_{0}$ to obtain $e_{0}:=\frac{p_{0}}{\left\|p_{0}\right\|}=\frac{1}{\sqrt{2}}$. Then

$\displaystyle e_{1}:=\dfrac{p_{1}-\langle{p_{1},e_{0}}\rangle e_{0}}{\left\|p_{1}-\langle{p_{1},e_{0}}\rangle e_{0}\right\|}=\sqrt{\frac{3}{2}}x$

Finally,

$\begin{array}{lcl}\displaystyle e_{2}:=\dfrac{p_{2}-\langle{p_{2},e_{1}}\rangle e_{1}-\langle{p_{2},e_{0}}\rangle e_{0}}{\left\|p_{2}-\langle{p_{2},e_{1}}\rangle e_{1}-\langle{p_{2},e_{0}}\rangle e_{0}\right\|}&=&\displaystyle\dfrac{p_{2}-\langle{p_{2},e_{0}}\rangle e_{0}}{\left\|p_{2}-\langle{p_{2},e_{0}}\rangle e_{0}\right\|}\\[2 em]\displaystyle&=&\dfrac{x^{2}-\frac{1}{2}\int_{-1}^{1}x^{2}dx}{\left\|p_{2}-\langle{p_{2},e_{0}}\rangle e_{0}\right\|}\\[2 em]\displaystyle&=&\dfrac{x^{2}-\frac{1}{3}}{\left(\int_{-1}^{1}(x^{2}-\frac{1}{3})^{2}dx\right)^{\frac{1}{2}}}\\[2 em]&=&\displaystyle\dfrac{x^{2}-\frac{1}{3}}{\left([\frac{1}{5}x^{5}-\frac{2}{9}x^{3}+\frac{1}{9}x]_{-1}^{1}\right)^{\frac{1}{2}}}\\[2 em]\displaystyle&=&\dfrac{x^{2}-\frac{1}{3}}{\sqrt{\frac{2}{5}-\frac{2}{9}}}\\[2 em]&=&\displaystyle\dfrac{3\sqrt{5}}{2\sqrt{2}}\left(x^{2}-\dfrac{1}{3}\right)\end{array}$

It’s worth mentioning that what we just did is essentially a truncated construction of the Legendre polynomials, one of the classical orthogonal polynomials. I use the word essentially because the Legendre polynomials $P_{n}$ are orthogonal, not orthonormal. Instead of normalizing in the Gram-Schmidt algorithm, they are standardized so that $P_{n}(1)=1$. Read about them!

Taking the orthogonal projection of $x^{3}$ onto the space $M$, we see that the solution is

$\begin{array}{lcl}\displaystyle\min_{a,b,c}\int_{-1}^{1}\left|x^{3}-a-bx-cx^{2}\right|^{2}dx&=&\displaystyle\langle{x^{3},e_{2}}\rangle e_{2}+\langle{x^{3},e_{1}}\rangle e_{1}+\langle{x^{3},e_{0}}\rangle e_{0}\\[2 em]\displaystyle&=&\left(\sqrt{\dfrac{3}{2}}\int_{-1}^{1}x^{4}dx\right)\sqrt{\dfrac{3}{2}}x\\[2 em]&=&\displaystyle\left(\dfrac{3}{2}\right)\left(\dfrac{2}{5}\right)x\\[2 em]&=&\displaystyle\dfrac{3}{5}x\end{array}$

The second part of exercise 13 hints at the use of duality, the connection between a Hilbert space and its space of bounded linear functionals (dual space), in solving optimization problems. Needless to say, duality is a major theme in mathematics.

Let $g\in L^{2}[-1,1]$ be a function that satisfies the constraints in the statement of the problem. Let $s$ denote the orthogonal projection of the polynomial $x^{3}$ onto the subspace $M$ of polynomials of degree at most $2$. Since $g\perp M$, we see that

$\displaystyle\langle{x^{3},g}\rangle=\langle{x^{3}-s,g}\rangle+\langle{s,g}\rangle=\langle{x^{3}-s,g}\rangle$

From the Cauchy-Schwarz inequality, we obtain the upper bound

$\displaystyle\left|\langle{x^{3}-s,g}\rangle\right|\leq\left\|x^{3}-s\right\|\left\|g\right\|=\left\|x^{3}-s\right\|$

But the upper bound is attained when $g=\frac{x^{3}-s}{\left\|x^{3}-s\right\|}$, which satisfies the constraints.