videos in 5.4

Author: surajrampure

05_matrices/04-null-space-rank-nullity.ipynb

@@ -106,6 +106,8 @@
     "\n",
     "The proof of this theorem is beyond the scope of our course. But, this is such an important theorem that it's sometimes called the **fundamental theorem of linear algebra**. It tells us, for one, that the dimension of the null space is equal to the number of columns minus the rank. \"Nullity\" is just another word for the dimension of the null space.\n",
     "\n",
+    "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/x1jZjbCfVYs?si=2q5SVMZ_LmHr1rJy\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen></iframe>\n",
+    "\n",
     "Let's see how it can be used in practice. Some of these examples are taken from Gilbert Strang's book."
    ]
   },
@@ -331,43 +333,6 @@
     "In both (and in the infinitely many other) cases, $\\begin{bmatrix} 3 \\\\ 1 \\end{bmatrix} \\in \\text{colsp}(A)$ and $\\begin{bmatrix} 1 \\\\ 3 \\end{bmatrix} \\in \\text{nullsp}(A)$.\n",
     ":::\n",
     "\n",
-    "### Example: Orthogonal Complements\n",
-    "\n",
-    "Suppose $\\vec r \\in \\text{colsp}(A^T)$ and $\\vec n \\in \\text{nullsp}(A)$, meaning $\\vec r$ is in the row space of $A$ and $\\vec n$ is in the null space of $A$.\n",
-    "\n",
-    "Prove that $\\vec r$ and $\\vec n$ must be orthogonal.\n",
-    "\n",
-    ":::{seealso} Solution\n",
-    ":class: dropdown\n",
-    "\n",
-    "The row space of $A$, $\\text{colsp}(A^T)$, is the set of all vectors $\\vec r$ where $\\vec r = A^T \\vec y$ for some $\\vec y \\in \\mathbb{R}^n$. Note that if $A$ is an $n \\times d$ matrix, then $A^T$ is a $d \\times n$ matrix, and $\\vec r$ is in $\\mathbb{R}^d$.\n",
-    "\n",
-    "The null space of $A$, $\\text{nullsp}(A)$, is the set of all vectors $\\vec n$ where $A \\vec n = \\vec 0$. Note that if $A$ is an $n \\times d$ matrix, then $\\vec n$ is in $\\mathbb{R}^d$.\n",
-    "\n",
-    "So, $\\vec r$ and $\\vec n$ are both in $\\mathbb{R}^d$, which means they exist in the same universe (they have the same number of components), and so we _can_ ask if they're orthogonal. (If they had different numbers of components, this question would be a non-starter.)\n",
-    "\n",
-    "In order to show that they're orthogonal, we need to show that their dot product is 0.\n",
-    "\n",
-    "$$\\begin{align*} \\vec r \\cdot \\vec n &= (A^T \\vec y) \\cdot \\vec n \\\\ &= \\underbrace{(A^T \\vec y)^T \\vec n}_{\\vec u \\cdot \\vec v = \\vec u^T \\vec v} \\\\ &= y^T \\underbrace{A \\vec n}_{\\vec 0} \\\\ &= y^T \\vec 0 \\\\ &= 0 \\end{align*}$$\n",
-    "\n",
-    "So, every vector in the row space of $A$ is orthogonal to every vector in the null space of $A$!\n",
-    ":::\n",
-    "\n",
-    ":::{attention} The row space and null space are orthogonal complements!\n",
-    "\n",
-    "Above, we proved that the row space and null space are orthogonal complements, in $\\mathbb{R}^d$. This means that every element in the row space is orthogonal to every element in the null space. The concept of an orthogonal complement was first introduced in [Chapter 4.3](../04_linear_independence/03-vector-spaces-basis-dimension.ipynb).\n",
-    "\n",
-    "It is also true that the column space and left null space are orthogonal complements, in $\\mathbb{R}^n$. Meaning, if\n",
-    "\n",
-    "$$\\vec x \\in \\text{colsp}(A), \\qquad \\vec y \\in \\text{nullsp}(A^T)$$\n",
-    "\n",
-    "then it must be the case that $\\vec x \\cdot \\vec y = 0$.\n",
-    "\n",
-    "To summarize:\n",
-    "- The row space, $\\text{colsp}(A^T)$, and null space, $\\text{nullsp}(A)$, are orthogonal complements; both are subspaces of $\\mathbb{R}^d$.\n",
-    "- The column space, $\\text{colsp}(A)$, and left null space, $\\text{nullsp}(A^T)$, are orthogonal complements; both are subspaces of $\\mathbb{R}^n$.\n",
-    ":::\n",
-    "\n",
     "### Example: Rank of $AB$ vs. Rank of $A$ or $B$\n",
     "\n",
     "Suppose $A$ is an $n \\times d$ matrix, and $B$ is a $d \\times p$ matrix.\n",
@@ -407,11 +372,52 @@
     "So intuitively, when we multiply two matrices, the rank of the resulting matrix can't be greater than the rank of either of the two matrices we started with, but it can \"drop\".\n",
     ":::\n",
     "\n",
-    "### Example: Rank of $X^TX$ 🚨\n",
+    "### Example: Orthogonal Complements 🎥\n",
+    "\n",
+    "Suppose $\\vec r \\in \\text{colsp}(A^T)$ and $\\vec n \\in \\text{nullsp}(A)$, meaning $\\vec r$ is in the row space of $A$ and $\\vec n$ is in the null space of $A$.\n",
+    "\n",
+    "Prove that $\\vec r$ and $\\vec n$ must be orthogonal.\n",
+    "\n",
+    "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/dcqA-6-vYA4?si=R_xevblhlyN2Ce7m\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen></iframe>\n",
+    "\n",
+    ":::{seealso} Solution\n",
+    ":class: dropdown\n",
+    "\n",
+    "The row space of $A$, $\\text{colsp}(A^T)$, is the set of all vectors $\\vec r$ where $\\vec r = A^T \\vec y$ for some $\\vec y \\in \\mathbb{R}^n$. Note that if $A$ is an $n \\times d$ matrix, then $A^T$ is a $d \\times n$ matrix, and $\\vec r$ is in $\\mathbb{R}^d$.\n",
+    "\n",
+    "The null space of $A$, $\\text{nullsp}(A)$, is the set of all vectors $\\vec n$ where $A \\vec n = \\vec 0$. Note that if $A$ is an $n \\times d$ matrix, then $\\vec n$ is in $\\mathbb{R}^d$.\n",
+    "\n",
+    "So, $\\vec r$ and $\\vec n$ are both in $\\mathbb{R}^d$, which means they exist in the same universe (they have the same number of components), and so we _can_ ask if they're orthogonal. (If they had different numbers of components, this question would be a non-starter.)\n",
+    "\n",
+    "In order to show that they're orthogonal, we need to show that their dot product is 0.\n",
+    "\n",
+    "$$\\begin{align*} \\vec r \\cdot \\vec n &= (A^T \\vec y) \\cdot \\vec n \\\\ &= \\underbrace{(A^T \\vec y)^T \\vec n}_{\\vec u \\cdot \\vec v = \\vec u^T \\vec v} \\\\ &= y^T \\underbrace{A \\vec n}_{\\vec 0} \\\\ &= y^T \\vec 0 \\\\ &= 0 \\end{align*}$$\n",
+    "\n",
+    "So, every vector in the row space of $A$ is orthogonal to every vector in the null space of $A$!\n",
+    ":::\n",
+    "\n",
+    ":::{attention} The row space and null space are orthogonal complements!\n",
+    "\n",
+    "Above, we proved that the row space and null space are orthogonal complements, in $\\mathbb{R}^d$. This means that every element in the row space is orthogonal to every element in the null space. The concept of an orthogonal complement was first introduced in [Chapter 4.3](../04_linear_independence/03-vector-spaces-basis-dimension.ipynb).\n",
+    "\n",
+    "It is also true that the column space and left null space are orthogonal complements, in $\\mathbb{R}^n$. Meaning, if\n",
+    "\n",
+    "$$\\vec x \\in \\text{colsp}(A), \\qquad \\vec y \\in \\text{nullsp}(A^T)$$\n",
+    "\n",
+    "then it must be the case that $\\vec x \\cdot \\vec y = 0$.\n",
+    "\n",
+    "To summarize:\n",
+    "- The row space, $\\text{colsp}(A^T)$, and null space, $\\text{nullsp}(A)$, are orthogonal complements; both are subspaces of $\\mathbb{R}^d$.\n",
+    "- The column space, $\\text{colsp}(A)$, and left null space, $\\text{nullsp}(A^T)$, are orthogonal complements; both are subspaces of $\\mathbb{R}^n$.\n",
+    ":::\n",
+    "\n",
+    "### Example: Rank of $X^TX$ 🎥\n",
     "\n",
     "Prove that $\\text{rank}(X^T X) = \\text{rank}(X)$ for any $n \\times d$ matrix $X$.\n",
     "\n",
-    "The matrix $X^TX$ is hugely important for our regression problem, and you'll also see in a homework that it helps define the **covariance** matrix of our data. (The 🚨 is to emphasize how important this example is.)\n",
+    "The matrix $X^TX$ is hugely important for our regression problem, and you'll also see in a homework that it helps define the **covariance** matrix of our data.\n",
+    "\n",
+    "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/hOyaHqGmO1I?si=DlSODDS57gRB_ZwB\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen></iframe>\n",
     "\n",
     ":::{seealso} Solution\n",
     ":class: dropdown\n",