Sheet 7

Exercise 1 (Portmonteau).

True of false? Justify your answer by proving or disproving the statement. Let $({\mu }_n,n\ge 1),\mu$ be finite measures (not necessarily probability measure) on $ℝ$.

1. If ${\mu }_n\to \mu$ weakly, then ${lim}_{n\to \mathrm{\infty }}{\mu }_n(ℝ)=\mu (ℝ)$.

   Solution.

   True. If ${\mu }_n\stackrel{(w)}{\to }\mu$, then ${\int }_{ℝ}1d{\mu }_n\to {\int }_{ℝ}1d\mu$. Alternatively: $ℝ$ is open and closed. ∎

2. If ${\mu }_n\to \mu$ weakly, then ${lim}_{n\to \mathrm{\infty }}{\mu }_n((-\mathrm{\infty },x])=\mu ((-\mathrm{\infty },x])$ for all $x\in ℝ$.

   Solution.

   False. Let $x_n\downarrow x$, then we have ${\delta }_{x_n}\to {\delta }_x$ by Sheet 6, Exercise 4(i). But for all $n\in \mathbb{N}$ we have ${\delta }_{x_n}((-\mathrm{\infty },x])=0$ while ${\delta }_x((-\mathrm{\infty },x])=1$. ∎

3. If ${lim}_{n\to \mathrm{\infty }}{\mu }_n((-\mathrm{\infty },x])=\mu ((-\mathrm{\infty },x])$ for all $x\in ℝ$, then ${\mu }_n\to \mu$ weakly.

   Solution.

   False. Let ${\mu }_n(A)={\int }_n^{n+1}{\mathrm{𝟙}}_{A}dx$ and $\mu (A)\equiv 0$. Then we have ${\mu }_n(ℝ)=1$ for all $n$ but $\mu (ℝ)=0$. But we also have ${\mu }_n((-\mathrm{\infty },x])\to 0=\mu ((-\mathrm{\infty },x])$.

   Here we abused the fact that tightness was not a given. Since we also have that any sequence which converges would converge to $\mu$, as $(-\mathrm{\infty },x]$ is a generator and uniquely determines the limit by Dynkin argumnets, we would get convergence with tightness (cf. Prohorov and Prop. 7.2.7) ∎

4. Suppose that ${\mu }_n,\mu \in {ℳ}_1(ℝ)$. If ${lim}_{n\to \mathrm{\infty }}{\mu }_n((-\mathrm{\infty },x])=\mu ((-\mathrm{\infty },x])$ for all $x\in ℝ$, then ${\mu }_n\to \mu$ weakly.

   Solution.

   True. Convergence of cumulative distribution functions implies convergence of measures (cf. 4.5.9). ∎

5. Let $f$ be a measurable bounded function on $ℝ$ which is $\mu$-a.e. continuous. If ${\mu }_n\to \mu$ weakly, then ${\int }_{ℝ}fd{\mu }_n\to {\int }_{ℝ}fd\mu$.

   Solution.

   Either we use the general continuous mapping theorem for almost surely continuous $f$ (comment after Theorem 7.2.5). I.e. ${\mu }_n\circ f\to \mu \circ f$ as $n$ goes to infinity. Then as $f$ is bounded there exists $B$ such that $|f(x)|\le B$ for all $x$ so using the transformation theorem (Thm. 3.1.16) we get

   ${\displaystyle {\int }_{ℝ}}fd{\mu }_n$ $={\displaystyle {\int }_{ℝ}}(f(x){\mathrm{𝟙}}_{|f(x)|\le B}^{\epsilon })d{\mu }_n(x)$

   $={\displaystyle {\int }_{ℝ}}x{\mathrm{𝟙}}_{|x|\le B}^{\epsilon }({\mu }_n\circ f)(\mathrm{d}x)\to {\displaystyle {\int }_{ℝ}}x{\mathrm{𝟙}}_{|x|\le B}^{\epsilon }(\mu \circ f)(\mathrm{d}x)={\displaystyle {\int }_{ℝ}}fd\mu$

   Or to do it properly (because this general continuous mapping theorem is no longer part of the lecture), we can use the Skorokhod trick from Stochastics 1. I.e. we use the cumulative distribution functions $F_n,F$ for ${\mu }_n,\mu$ and $U\sim 𝒰[0,1]$ to get

   $X_n:=F_n^{-1}(U)\to F^{-1}(U)=:X\mathit{\hspace{1em}}\text{a.s.}$

   with $X_n\sim {\mu }_n$ and $X\sim \mu$. Since $f$ is continuous $\mu$-a.e. we get $f(X_n)\to f(X)$ a.s. and therefore in distribution, which means that ${\mu }_n\circ f\to \mu \circ f$. We finish with the same argument as before. ∎

Exercise 2 (Tightness).

Let $({\mu }_n,n\ge 1)$ be finite measures on $ℝ$. Show that the family is tight, if and only if there exists a non-negative measurable function $f$ with ${lim}_{|x|\to \mathrm{\infty }}f(x)=\mathrm{\infty }$, such that

Solution.

$\Leftarrow$: Take some $ϵ>0$, and define for such an $f$

Since $f(x)\to \mathrm{\infty }$ we know there exists $M$ such that $f(x)>\frac{C}{\epsilon }$ for all $|x|>M$. Therefore for all $n$

${\mu }_n\left({[-M,M]}^c\right)\cdot {\displaystyle \frac{C}{\epsilon }}\le {\displaystyle {\int }_{ℝ}}fd{\mu }_n\le C$

Hence, ${sup}_{n\ge 1}{\mu }_n\left({[-M,M]}^c\right)\le \epsilon$.

$\Rightarrow$: Suppose that ${\mu }_n$ is tight. Then for all $j>0$ exists $K_j$, such that . Without loss of generality $K_j=[-k_j,k_j]$ with $0\le k_1\le \mathrm{\dots }{k}_j$ and we define $f(x)=j$ for all $x\in K_{j+1}\setminus K_j$. This implies $f(x)\to \mathrm{\infty }$ for $|x|\to \mathrm{\infty }$ and

${\displaystyle {\int }_{ℝ}}fd{\mu }_n$ $={\displaystyle \sum _{j\ge 1}}j\cdot {\mu }_n\left(K_{j+1}\setminus K_j\right)$

$\le {\displaystyle \sum _{j\ge 1}}j\cdot {\mu }_n(ℝ\setminus K_j)$

Exercise 3 (Totally Bounded).

Let $(E,d)$ be a metric space and $A\subset E$. Compare Lemma 4.3.4.  with this exercise.

1. Suppose that, for all $ϵ>0$, there exists finitely many points $x_1,\mathrm{\dots }{x}_n\in E$ (not necessarily in $A$) such that $A\subseteq {\bigcup }_{k=1}^n(B_{ϵ,k})$. Show that $A$ is totally bounded.

   Solution.

   For all $\epsilon >0$ we have $x_1,\mathrm{\dots },x_n\in E$ such that

   $A\subseteq {\displaystyle \bigcup _{k=1}^n}B(x_k,\frac{\epsilon }{2}).$

   Take $y_k\in B(x_k,\frac{\epsilon }{2})\cap A$, (if it was empty it was not needed to cover $A$ so it could be discarded). Then we have $B(x_k,\frac{\epsilon }{2})\subseteq B(y_k,\epsilon )$ because for any $z$ in the first set we have

   $d(z,y_k)\le d(z,x_k)+d(x_k,y_k)\le \epsilon$

   which finally implies

   $A$ $\subseteq {\displaystyle \bigcup _{k=1}^n}B(x_k,\frac{\epsilon }{2})\subseteq {\displaystyle \bigcup _{k=1}^n}B(y_k,\epsilon ).\mathit{∎}$

2. Let $E=\mathbb{Q}$, endowed with the usual Euclidean metric. Show that, $A=\mathbb{Q}\cap (0,1)$ is totally bounded, but not relatively compact. Why is this not a contradiction to Sheet 6, Exercise 2(iv)?

   Solution.

   Since $[0,1]$ is compact in $ℝ$ it is also totally bounded. Therefore $A\subseteq [0,1]$ is totally bounded as a subset, because the epsilon coverings still contain all the previous elements. We just have to select new mid points, but this is no issue because $\mathbb{Q}$ is dense and we can start with $\epsilon /2$.

   The closure of $A$ in $\mathbb{Q}$ is $A\cap [0,1]$, which is not complete. There are Cauchy sequences which do not converge in $A$ and therefore have no converging subseqence. It can therefore not be compact.

   It is not a contradiction to Sheet 6, Exercise 2(iv) because the space $E$ is not complete. In other words: compactness is totally boundedness + completeness. ∎

Exercise 4 (Weak Convergence of Dirac Measures).

Let $x_n\in E$, where $(E,d)$ is a separable and complete metric space.

1. Prove that ${\delta }_{x_n}$ is tight, if and only if ${(x_n)}_{n\in \mathbb{N}}$ is totally bounded in $E$.

   Solution.

   Assume that ${\delta }_{x_n}$ are tight, then there exists a compact set $K$, such that

   As the Dirac measure can only be zero or one, this implies

   $\underset{n\in \mathbb{N}}{sup}{\delta }_{x_n}(K^c)=0$

   and therefore $x_n\in K$ for all $n\in \mathbb{N}$. But $K$ is totally bounded as a compact set, therefore the ${(x_n)}_{n\in \mathbb{N}}$ are totally bounded.

   For the opposite direction note that the closure of a totally bounded set is still totally bounded by Sheet 6, Exercise 2(iv). So the closure $K$ is compact and includes all $x_n$. Therefore

   $\underset{n\in \mathbb{N}}{sup}{\delta }_{x_n}(K^c)=0,$

   and the dirac measures are tight. ∎

2. For $E=ℝ$ suppose that ${\delta }_{x_n}$ converges weakly to some $\mu$. Show that, there exists $x\in ℝ$, such that $x_n\to x$ and therefore $\mu ={\delta }_x$.

   Hint.

   Assume you could use the identity as a test function in weak convergence. While this does not work (the identity is not bounded) it provides some intuition how the actual proof would work. Then try an invertible sigmoid function instead (e.g. $f\mathit{}\mathrm{(}x\mathrm{)}\mathrm{=}\frac{\mathrm{1}}{\mathrm{1}\mathrm{+}{e}^{\mathrm{-}x}}$)

   Solution.

   Let $X_n\sim {\delta }_{x_n}$ and $X\sim \mu$. Then we have for $f(x)=\frac{1}{1+e^{-x}}\in C_b(ℝ)$

   $\underset{n\to \mathrm{\infty }}{lim}\underset{=:y_n}{\underbrace{f(x_n)}}=\underset{n\to \mathrm{\infty }}{lim}𝔼[f(X_n)]=𝔼[f(X)]=:y$

   Now if $y=1$, then $x_n=f^{-1}(y_n)\to \mathrm{\infty }$. Or for $y=0$ $x_n\to -\mathrm{\infty }$ respectively. Both would imply by (i) that ${\delta }_{x_n}$ is not tight. But this would contradict Theorem 7.3.4. Therefore $y\in (0,1)$. And by continuity of $f^{-1}$ on $(0,1)$, we can define

   $\underset{n\to \mathrm{\infty }}{lim}{x}_n=\underset{n\to \mathrm{\infty }}{lim}{f}^{-1}(y_n)=f^{-1}(y)=:x.$

   By Sheet 6, Exercise 5 (i), we therefore have ${\delta }_{x_n}\to {\delta }_x$ weakly which implies ${\delta }_x=\mu$. ∎

Exercise 5 (Arzelà-Ascoli).

We are going to prove the Arzelà-Ascoli theorem in this exercise. While it is usually formulated with (relative) compactness, we have already seen this is equivalent to total boundedness in a complete space.

Definition (Equicontinuity).

Let $\mathrm{(}E\mathrm{,}d\mathrm{)}\mathrm{,}\mathrm{(}F\mathrm{,}d\mathrm{)}$ be metric spaces and $ℱ$ a set of functions from $E$ to $F$. Then $ℱ$ is equicontinuous iff

In other words: $\delta$ is independent of $f\mathrm{\in }ℱ$ (and also of $x\mathrm{,}y$, i.e. we also have uniform continuity).

1. Prove that any finite subset of $C(K)$ for a compact space $K$ is equicontinuous.

   Solution.

   Let $\{f_1,\mathrm{\dots },f_d\}\subseteq C(K)$. As any continuous function is uniformly continuous on a compact set, we have

   With $\delta :={\min }_{i=1,\mathrm{\dots },d}{\delta }_i$ we get equicontinuity. ∎

2. Prove that total boundedness of a subset of $(C(K),\parallel \cdot {\parallel }_{\mathrm{\infty }})$ implies equicontinuity.

   Solution.

   Let $ℱ\subseteq C(K)$ be a totally bounded set and choose some $ϵ>0$. Then we have by total boundedness $f_1,\mathrm{\dots },f_d$ such that for all $f\in ℱ$ exists an $f_i$ with

   And since the $f_1,\mathrm{\dots },f_d$ are equicontinuous by (i), we can find some $\delta >0$, such that

   Putting things together we have for all $f\in ℱ$, all $x,y\in K$ with

   $|f(x)-f(y)|$ $\le \underset{\le {\parallel f-f_i\parallel }_{\mathrm{\infty }}\le \frac{ϵ}{4}}{\underbrace{|f(x)-f_i(x)|}}+\underset{\le \frac{ϵ}{2}}{\underbrace{|f_i(x)-f_i(y)|}}+\underset{\le {\parallel f-f_i\parallel }_{\mathrm{\infty }}\le \frac{ϵ}{4}}{\underbrace{|f_i(y)-f(y)|}}$

   $\le ϵ.\mathit{∎}$

3. Let $ℱ\subseteq C(K)$ be a pointwise bounded set, i.e.

   Prove that for any finite set $\{x_1,\mathrm{\dots },x_d\}\subseteq K$, the set

   $\mathrm{\Phi }=\{(f(x_1),\mathrm{\dots },f(x_d)):f\in ℱ\}$

   is totally bounded.

   Solution.

   $\mathrm{\Phi }$ is bounded in $({ℝ}^d,\parallel \cdot {\parallel }_{\mathrm{\infty }})$ by ${\max }_{i=1}^{d}M(x_i)$. Therefore it is totally bounded as this is equivalent on ${ℝ}^d$. ∎

4. Prove that pointwise boundedness and equicontinuity of $ℱ\subseteq C(K)$ imply total boundedness (with regard to the sup-norm $\parallel \cdot {\parallel }_{\mathrm{\infty }}$).

   Hint.

   Use equicontinuity to extend (iii) to all points. Do not forget that $K$ is totally bounded!

   Solution.

   For a given $ϵ>0$ we want to find an $ϵ$ covering. Since $ℱ$ is equicontinuous, we can select $\delta >0$, such that for all we have

   $|f(x)-f(y)|\le {\displaystyle \frac{ϵ}{4}}\mathit{\hspace{1em}}\forall f\in ℱ.$ (1)

   Since $K$ is totally bounded, we can find a $\delta$ covering, i.e. there exist $x_1,\mathrm{\dots },x_d$, such that for all $x\in K$ there exists $k$ with

   $d(x,x_k)\le \delta .$ (2)

   Due to total boundedness of $\mathrm{\Phi }$ by (iii) there exist $f_1,\mathrm{\dots },f_m$ such that for any $\varphi (f)\in \mathrm{\Phi }$ we can find $i$ with

   (3)

   So take any $f\in ℱ$ and choose $f_i$ such that (3) is satisfied. Then for all $x\in K$ there exists an $x_k$ by (2) such that and hence

   $|f(x)-f_i(x)|\le \underset{\stackrel{(\mathit{\text{<a href="\#S0.E1" title="(1) ‣ item (iv) ‣ Exercise 5 (Arzelà-Ascoli). ‣ Sheet 7" class="ltx\_ref ltx\_font\_italic" style="font-size:70\%;"><span class="ltx\_text ltx\_ref\_tag">1</span></a>}})}{\le }\frac{ϵ}{4}}{\underbrace{|f(x)-f(x_k)|}}+\underset{\le {\parallel \varphi (f)-\varphi (f_i)\parallel }_{\mathrm{\infty }}\le \frac{ϵ}{2}}{\underbrace{|f(x_k)-f_i(x_k)|}}+\underset{\stackrel{(\mathit{\text{<a href="\#S0.E1" title="(1) ‣ item (iv) ‣ Exercise 5 (Arzelà-Ascoli). ‣ Sheet 7" class="ltx\_ref ltx\_font\_italic" style="font-size:70\%;"><span class="ltx\_text ltx\_ref\_tag">1</span></a>}})}{\le }\frac{ϵ}{4}}{\underbrace{|f_i(x_k)-f_i(x)|}}$

   As $x$ was arbitrary we have

   ${\parallel f-f_i\parallel }_{\mathrm{\infty }}\le ϵ.$

   Therefore $f_1,\mathrm{\dots },f_m$ are an $ϵ$ covering of $ℱ$, proving its total boundedness. ∎

Putting (ii) and (iv) together we therefore have

Theorem (Arzelà-Ascoli).

Let $K\mathrm{\subseteq }E$ be totally bounded in a metric space $\mathrm{(}E\mathrm{,}d\mathrm{)}$. A set of continuous functions mapping to $ℝ$, $ℱ\mathrm{\subseteq }C\mathit{}\mathrm{(}K\mathrm{)}$, is totally bounded with regard to the sup-norm iff it is pointwise bounded and equicontinuous.

1. Deduce that $𝒫:=\{x\mapsto x^n:n\in \mathbb{N}\}\subseteq C([0,1])$ is bounded but not totally bounded.

   Solution.

   Since $p(x)\in [0,1]$ for all $p\in 𝒫$, the set is bounded. To show that it is not totally bounded, we will show it is not equicontinuous. Take any $ϵ>0$ and choose some $\delta >0$. Then there exists a polynomial such that

   $|p(1)-p(1-\delta )|=1-{(1-\delta )}^n>ϵ$

   for some $n\in \mathbb{N}$. Therefore the set $𝒫$ can not be equicontinuous. ∎