A. Consider the matrix
Find the range, nullspace, and rank of .
Find an SVD of .
Determine the set of solutions to the linear equation , with
B. Consider the matrix
What is an SVD of ? Express it as , with the diagonal matrix of singular values ordered in decreasing fashion. Make sure to check all the properties required for .
Find the semi-axis lengths and principal axes (minimum and maximum distance and associated directions from to the center) of the ellipsoid Hint: Use the SVD of to show that every element of is of the form for some element in . That is, . (In other words the matrix maps into the set .) Then analyze the geometry of the simpler set .
What is the set when we append a zero vector after the last column of , that is is replaced with ?
Same question when we append a row after the last row of , that is, is replaced with . Interpret geometrically your result.
C. The image on the left shows a matrix of pixel values. The lines indicate values; at each intersection of lines the corresponding matrix element is . All the other elements are zero.
D. The Orthogonal Procrustes problem is a problem of the form
where denotes the Frobenius norm, and the matrices , are given. Here, the matrix variable is constrained to have orthonormal columns. When , the problem can be interpreted geometrically as seeking a transformation of points (contained in ) to other points (contained in ) that involves only rotation.
Show that the solution of the Procrustes problem above can be found via the SVD of the matrix .
Derive a formula for the answer to the constrained least-squares problem with , given.
E. We consider a set of data points , . We seek to find a line in such that the sum of the squares of the distances from the points to the line is minimized. To simplify, we assume that the line goes through the origin.
Consider a line that goes through the origin , where is given. (You can assume without loss of generality that .) Find an expression for the projection of a given point on .
Now consider the points and find an expression for the sum of the squares of the distances from the points to the line .
Explain how you would find the line via the SVD of the matrix .
How would you address the problem without the restriction that the line has to pass through the origin?
Solve the same problems as previously by replacing the line by an hyperplane.
F. Consider the matrix formed as , with with a vector chosen randomly in . In addition we define with again chosen randomly in . We consider the associated least-squares problem
What is the rank of ?
Apply the least-squares formula . What is the norm of the residual vector, ?
Express the least-squares solution in terms of the SVD of . That is, form the pseudo-inverse of and apply the formula . What is now the norm of the residual?
Interpret your results.
G. Consider a least-squares problem
where the data matrix has rank one.
Is the solution unique?
Show how to reduce the problem to one involving one scalar variable.
Express the set of solutions in closed-form.