Big Rudin: Chapter 4, Exercise 14

A week or so ago, a commenter requested that solutions to exercises 14 and 15 of Walter Rudin’s Real and Complex Analysis. The commenter failed to specify which chapter; but the comment was to a post for an exercise in chapter 4 of Baby Rudin, so I am guessing it’s chapter 4. It’s been an exhausting week at work, so I only have a solution to exercise 14. I’ll probably have exercise 14 ready later this weekend.

Underlying both exercises is the subject of orthogonal polynomials, which I encourage the reader to explore after solving the exercises; however, I will not discuss it here. We will make do with basic Hilbert space results and integral calculus.

The first result we need is on the orthogonal projection of an element x onto a closed, convex set E of a Hilbert space. Specifically, there is a unique element s of E which minimizes the distance between x and E. If E is a subspace, then s also satisfies x-s\perp M. Moreover, this element s can be computed using generalized Fourier series.

Lemma 1. Let H be a Hilbert space, and let \left\{e_{\alpha}:\alpha\in A\right\} be an orthonormal set, possibly uncountable, in H. Let F be a finite subset of A, and define a subspace M:=\text{span}\left\{e_{\alpha}:\alpha\in F\right\}. Then

\displaystyle\left\|x-s_{F}\right\|\leq\left\|x-s\right\|,\indent\forall s\in M


\displaystyle s_{F}:=\sum_{\alpha\in F}\langle{x,e_{\alpha}}\rangle e_{\alpha}

Equality holds if and only if s=s_{F}.

We can drop the finiteness condition imposed on F with a bit more work, but we will not need the full result for our purposes. I leave generalizing Lemma 1 as an exercise to the reader.

Proof. Since F is finite, we can enumerate the set \left\{e_{\alpha}:\alpha\in F\right\} by \left\{e_{i}:1\leq i\leq n\right\}, for some positive integer n. Observe that, for 1\leq j\leq n,

\begin{array}{lcl}\displaystyle\left\langle{x-s_{F},e_{j}}\right\rangle&=&\displaystyle\langle{x,e_{j}}\rangle-\langle{s_{F},e_{j}}\rangle\\[2 em]\displaystyle&=&\langle{x,e_{j}}\rangle-\sum_{i=1}^{n}\langle{x,e_{i}}\rangle\langle{e_{i},e_{j}}\rangle\\[2 em]\displaystyle&=&\langle{x,e_{j}}\rangle-\langle{x,e_{j}}\rangle\\[2 em]\displaystyle&=&0\end{array}

It follows from the sesquilinearity of the inner product that x-s_{F}\perp M. Since s_{F}-s\in M, it follows from the Pythagorean theorem that


It is evident that the inequality is strict unless s=s_{F}. \Box

Note that in the preceding lemma, s_{F} is the unique solution to \min_{s\in M}\left\|x-s\right\|^{2}. But we can say more: s_{F} is the unique element s\in M with the property that x-s\perp M. To see this, observe that any such element s, by the Pythaogorean theorem, satisfies


From our earlier work, we conclude that s=s_{F}.

Consider the space M of real polynomials with degree at most 2. Define polynomials p_{0}, p_{1}, and p_{2} by

\displaystyle p_{0}(x):=1,\indent p_{1}(x):=x,\indent p_{2}(x):=x^{2}

for all x\in[-1,1]. We use the Gram-Schmidt algorithm to construct an orthonormal basis \left\{e_{0},e_{1},e_{2}\right\} out of the basis \left\{p_{0},p_{1},p_{2}\right\} for the space M. We normalize p_{0} to obtain e_{0}:=\frac{p_{0}}{\left\|p_{0}\right\|}=\frac{1}{\sqrt{2}}. Then

\displaystyle e_{1}:=\dfrac{p_{1}-\langle{p_{1},e_{0}}\rangle e_{0}}{\left\|p_{1}-\langle{p_{1},e_{0}}\rangle e_{0}\right\|}=\sqrt{\frac{3}{2}}x


\begin{array}{lcl}\displaystyle e_{2}:=\dfrac{p_{2}-\langle{p_{2},e_{1}}\rangle e_{1}-\langle{p_{2},e_{0}}\rangle e_{0}}{\left\|p_{2}-\langle{p_{2},e_{1}}\rangle e_{1}-\langle{p_{2},e_{0}}\rangle e_{0}\right\|}&=&\displaystyle\dfrac{p_{2}-\langle{p_{2},e_{0}}\rangle e_{0}}{\left\|p_{2}-\langle{p_{2},e_{0}}\rangle e_{0}\right\|}\\[2 em]\displaystyle&=&\dfrac{x^{2}-\frac{1}{2}\int_{-1}^{1}x^{2}dx}{\left\|p_{2}-\langle{p_{2},e_{0}}\rangle e_{0}\right\|}\\[2 em]\displaystyle&=&\dfrac{x^{2}-\frac{1}{3}}{\left(\int_{-1}^{1}(x^{2}-\frac{1}{3})^{2}dx\right)^{\frac{1}{2}}}\\[2 em]&=&\displaystyle\dfrac{x^{2}-\frac{1}{3}}{\left([\frac{1}{5}x^{5}-\frac{2}{9}x^{3}+\frac{1}{9}x]_{-1}^{1}\right)^{\frac{1}{2}}}\\[2 em]\displaystyle&=&\dfrac{x^{2}-\frac{1}{3}}{\sqrt{\frac{2}{5}-\frac{2}{9}}}\\[2 em]&=&\displaystyle\dfrac{3\sqrt{5}}{2\sqrt{2}}\left(x^{2}-\dfrac{1}{3}\right)\end{array}

It’s worth mentioning that what we just did is essentially a truncated construction of the Legendre polynomials, one of the classical orthogonal polynomials. I use the word essentially because the Legendre polynomials P_{n} are orthogonal, not orthonormal. Instead of normalizing in the Gram-Schmidt algorithm, they are standardized so that P_{n}(1)=1. Read about them!

Taking the orthogonal projection of x^{3} onto the space M, we see that the solution is

\begin{array}{lcl}\displaystyle\min_{a,b,c}\int_{-1}^{1}\left|x^{3}-a-bx-cx^{2}\right|^{2}dx&=&\displaystyle\langle{x^{3},e_{2}}\rangle e_{2}+\langle{x^{3},e_{1}}\rangle e_{1}+\langle{x^{3},e_{0}}\rangle e_{0}\\[2 em]\displaystyle&=&\left(\sqrt{\dfrac{3}{2}}\int_{-1}^{1}x^{4}dx\right)\sqrt{\dfrac{3}{2}}x\\[2 em]&=&\displaystyle\left(\dfrac{3}{2}\right)\left(\dfrac{2}{5}\right)x\\[2 em]&=&\displaystyle\dfrac{3}{5}x\end{array}

The second part of exercise 13 hints at the use of duality, the connection between a Hilbert space and its space of bounded linear functionals (dual space), in solving optimization problems. Needless to say, duality is a major theme in mathematics.

Let g\in L^{2}[-1,1] be a function that satisfies the constraints in the statement of the problem. Let s denote the orthogonal projection of the polynomial x^{3} onto the subspace M of polynomials of degree at most 2. Since g\perp M, we see that


From the Cauchy-Schwarz inequality, we obtain the upper bound


But the upper bound is attained when g=\frac{x^{3}-s}{\left\|x^{3}-s\right\|}, which satisfies the constraints.

This entry was posted in math.CA, Textbook Solutions and tagged , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s