Cauchy-schwartz inequality and angles
Hyperplanes and half-spaces
The scalar product (or, dot product) between two vectors is the scalar denoted , and defined as
The scalar product is sometimes denoted . The motivation for our notation above will come later, when we define the matrix-vector product.
We say that the vectors are orthogonal if .Matlab syntax
>> x = [1; 2; 3]; y = [4; 5; 6]; >> scal_prod = x'*y;
Measuring the size of a scalar value is unambiguous — we just take the magnitude (absolute value) of the number. However, when we deal with higher dimensions, and try to define the notion of size, or length, of a vector, we are faced with many possible choices.
Norms are real-valued functions that satisfy a basic set of rules that a sensible notion of size should involve. You can consult the formal definition of a norm here. In this course, we focus on the following three popular norms for a vector :
The Euclidean norm: corresponds to the usual notion of distance in two or three dimensions. The set of points with equal -norm is a circle (in 2D), a sphere (in 3D), or a hyper-sphere in higher dimensions.
The -norm: corresponds to the distance travelled on a rectangular grid to go from one point to another.
The -norm: is useful in measuring peak values.
>> x = [1; 2; -3]; >> r2 = norm(x,2); % l2-norm >> r1 = norm(x,1); % l1 norm >> rinf = norm(x,inf); % l-infty norm
A given vector will in general have different ‘‘lengths" under different norms. For example, the vector yields , , and .
The Cauchy-Schwartz inequality allows to bound the scalar product of two vectors in terms of their Euclidean norm.
Cauchy-Schwartz inequality: For any two vectors , we have with equality if and only if are collinear.
When none of the vectors involved is zero, we can define the corresponding angle as such that The notion above generalizes the usual notion of angle between two directions in two dimensions, and is useful in measuring the similarity (or, closeness) between two vectors. When the two vectors are orthogonal, that is, , we do obtain that their angle is .
The Cauchy-Schwartz inequality can be generalized to other norms, using the concept of dual norm.
A basis is said to be orthogonal if if . If in addition, , we say that the basis is orthonormal.
Example: An orthonormal basis in . The collection of vectors , with forms an orthonormal basis of .
A hyperplane is a set described by a single affine equality. Precisely, an hyperplane in is a set of the form where , , and are given. When , the hyperplane is simply the set of points that are orthogonal to ; when , the hyperplane is a translation, along direction , of that set.
Hyperplanes are affine sets, of dimension (see the proof here). Thus, they generalize the usual notion of a plane in . Hyperplanes are very useful because they allows to separate the whole space in two regions. The notion of half-space formalizes this.
A half-space is a subset of defined by a single affine inequality. Precisely, an half-space in is a set of the form where , , and are given.
Based on the notion of angle between two vectors, we can understand the meaning of an inequality of the form , where and are given.
Let us examine the case when . The condition means that the angle between and is acute, while means the angle is obtuse. The set defines a halfspace with boundary passing through , and outward vector . When , the half-space is a translated version of th