Affine sets Linear Inequality Systems Definition 6.12 The vectors v 1, v 2,..., v k are affinely independent if v 2 v 1,..., v k v 1 is linearly independent; affinely dependent, otherwise. We first check the subtracting vector is irrelevant in the definition: v 2 v 1, v 3 v 1,..., v k v 1 are linearly independent v 1 v 2, v 3 v 2,..., v k v 2 are linearly independent. (Check it.) It is also easy to see that affine independence is invariant with a translation. Proposition 6.13 The vectors v 1, v 2,..., v k are affinely independent only λ = 0 satisfies the following. The converse is also true. λ 1 v 1 + + λ k v k = 0 λ 1 + + λ k = 0. (6.15) (6.15) is equivalent to that the following vectors are linearly independent. [ ] [ ] [ ] v1 v2 vk,,,. 1 1 1 Optimization Lab. 25th March 2018 42 / 50
Affine sets Linear Inequality Systems Exercise 6.14 Linearly independent vectors are also affinely independent. If we translate, by w / S, a basis of a subspace S, and add w to it, then the resulting set is a set of affinely independent vectors. Therefore, the maximum number of affinely independent vectors from S + w is dim(s) + 1. But it can not exceed dim(s) + 1 (why?). Proposition 6.15 The maximum number of affinely independent vectors in S + w is dim S + 1. Optimization Lab. 25th March 2018 43 / 50
Affine sets Linear Inequality Systems FYI Let L and L w with w L, respectively, be an affine space and its subspace. Then {v 1, v 2,..., v k } L {v 2 v 1,..., v k v 1 } L w. Example 6.16 In the figure, v 1, v 2, v 3 are linearly as well as affinely idependent. Also 0, v 2 v 1, v 3 v 1 is affinely independent but not linearly independent. Optimization Lab. 25th March 2018 44 / 50
Affine sets Linear Inequality Systems FYI Proposition 6.17 If an affine space does not contain the origin, its affinely independent vectors are linearly independent. (Therefore, their maximum number is the same as the dimension of the subspace plus 1.) Proof: Let s consider a subspace S and an affine space S + w for some w / S. Assume v 1, v 2,..., v k are affinely independent vectors in S + w and α 1 v 1 + α 2 v 2 + + α k v k = 0. Then α 1 (v 1 w) + α 2 (v 2 w) + + α k (v k w) = (α 1 + + α k )w. (6.16) While v 1 w,..., v k w are in S, w is not. Therefore (6.16) is possible only if α 1 + + α k = 0. Since v i s are affinely independent, we have, by Exercise 6.13, α 1 = = α k = 0. Definition 6.18 The dimension of an affine space L is defined to be the maximum number of affinely independent vectors from L minus 1. Optimization Lab. 25th March 2018 45 / 50
Affine sets Linear Inequality Systems FYI Suppose w 1,, w k are affinely dependent vectors. Then there is β = (β 1,..., β k ) 0: β 1 w 1 + β 1 w 1 + + β k w k = 0, β 1 + β 2 + + β k = 0. (6.17) We may assume β 1 0. Then, if we write µ i := β i /β 1, (6.17) becomes w 1 = µ 2 w 2 + + µ k+1 w k+1, µ 2 + + µ k+1 = 1 (6.18) Hence w 1 is an affine combination of w 2,..., w k. We can see that Proposition 6.19 The vectors w 1,, w k are affinely dependent if and only there is i such that w i is an affine combination of the others. Optimization Lab. 25th March 2018 46 / 50
Affine sets Linear Inequality Systems FYI A subspace S is the set of linear combinations of a maximal linearly independent set of vectors it contains. Likewise, an affine set L is the set of affine combinations of a maximal affinely independent set of vectors it has. Proposition 6.20 A k-dimensional affine set is the set of affine combinations of its (k + 1) affinely independent vectors. Definition 6.21 The dimension of a convex set is defined to be the the maximum number of its affinely independent vectors minus 1. Optimization Lab. 25th March 2018 47 / 50
Valid inequalities Linear Inequality Systems Definition 7.1 Valid inequalities ( ) If the half space {x : a T x b} includes S, a T x b is called a valid inequality for S. Definition 7.2 Valid inequality and supporting hyperplane Supporting hyperplanes ( ) If a T x b is valid for S and S {x : a T x = b} =, H = {x : a T x = b} is called a supporting hyperplane of S. Optimization Lab. 25th March 2018 48 / 50
Valid inequalities Linear Inequality Systems Definition 7.3 Separating hyperplane ( ) H = {x : a T x = b} is called a separating hyperplane of a set S R n and a point z R n S if a T x b is valid for S but not for z. A separating hyperplane of S and z. Optimization Lab. 25th March 2018 49 / 50
Valid inequalities Linear Inequality Systems Theorem 7.4 Separating heperplane theorem ( ) If S is a closed set and z / S, then there is a separating hyperplane of z and S. The theorem implies that any closed convex set C is the intersection of the half spaces separating C and the points not in C. We can prove Farkas Lemma by using Separating hyperplane theorem. Optimization Lab. 25th March 2018 50 / 50
Polyhedra Optimization Lab. IE department Seoul National University 23rd April 2018 Optimization Lab. 23rd April 2018 1 / 38
Polyhedra Definition 1.1 We call the feasible solution set of a linear system Ax b (A R m n ) a polyhedron. In other words, a polyhedron is the intersection of a finite number of half spaces. A = 0 3 2 0 3 2 2 0 3 2 0 3 3 2 0 3 2 0 0 3 2 0 3 2 2 0 3 2 0 3 3 2 0 3 2 0, b = 5 6 5 4 5 5 6 5 6 5 4 6. Dodecahedron ( 12 ) - from The representation of polyhedra by polynomial inequalities M Grötschel and M. Henk, (2004). Optimization Lab. 23rd April 2018 2 / 38
Polyhedra Regular dodecahedron ( 12 ) φ = 1+ 5 2, α > 0, ±φ 2 x ± φy α, ±φ 2 y ± φz α, ±φ 2 z ± φx α. Optimization Lab. 23rd April 2018 3 / 38
Polyhedra A polyhedron P is said to be bounded if it does not contain any half line (or unbounded ray), or equivalently if P has vector lower and upper bounds: there are l, u R n such that P {x : l x u}. If every point of P = {x Ax b} satisfies an inequality A i x b i for some i with equality, we call it an implicit equality. The subsystem of implicit equalities of Ax b is called the implicit equality subsystem and denoted by A = x b =. Exercise 1.2 There is a point of P satisfying each of the remaining subsystem with strict inequality, >. Optimization Lab. 23rd April 2018 4 / 38
Polyhedra Lemma 1.3 If an affine set L R n has a point u satisfying D T u > d, then L and L {x : D T x d} has the same dimension. Proof: By the assumption, there is ɛ > 0 such that every point of L B ɛ (u) satisfies D T x d. (The figure shows the case when Dx d is a T x β.) Let v 1, v 2,..., v k be affinely independent vectors in L. Since L is affine and hence closed under translation between its points, the following vectors are also in L: u, u+ ɛ v 2 v 1 2 v, 2 v 1..., u + ɛ v k v 1 2 v k v 1. They are also affinely independent vectors each contained in L B ɛ (u). This implies L and L {x : D T x d} have the same dimension. Proposition 1.4 P has the same dimension as the affine set A = x = b = : dim(p ) = n r(a = ). Optimization Lab. 23rd April 2018 5 / 38
Faces Definition 2.1 By a face of a polyhedron P, we mean its intersection with a supporting hyperplane. If there is an implicit equality a T x β, H = {x a T x = β} is a supporting hyperplane which includes P. Hence P itself is a face. By a proper face, we mean a face other than P. Proposition 2.2 Face-subsystem proposition ( - ) Let F be a face of P = {x : Ax b}. Then there is a subsystem A x b of Ax b such that F = {x P : A x = b }. Conversely, if P {x : A x = b } for a subsystem A x b, F := P H is a face of P. (Will call A x = b } in the proposition a face subsystem of F.) Optimization Lab. 23rd April 2018 6 / 38
Faces Proof: Let F be a face of P and c T x = d be the supporting hyperplane. I.e. F = {x P : c T x = d} and c T x d for every x P. This means exactly that F is the set of optimal solutions of LP min{c T x : Ax b}. By strong duality, there is dual optimal solution y 0. Let A x b be a subsystem of Ax b corresponding to positive components of y. Then by complementary slackness theorem, an element x of P is optimal if and only if A x = b. Hence F = {x P : A x = b }. Conversely, suppose there is a subsystem A x b such that F := P {x : A x = b } =. Suppose we have taken the sum of the equations of A x = b to get equation c T x = δ. The equation is satisfied by every element of F and hence F P {x : c T x = δ}. For the reverse inclusion, consider any x P F. Then by definition at least one inequality from A x b is satisfied with >. Thus any point of P satisfying c T x = δ is also in F : F P {x : c T x = δ}. Assume c 0 so that c T x = δ is a hyperplane. Then by definition F is a face of P. For the case when c = 0, see Exercise 2.3. Optimization Lab. 23rd April 2018 7 / 38
Faces Exercise 2.3 Complete the proof for the case c = 0 (so that δ = 0). (Hint: P is a face.) A face subsystem is, however, not unique in general. Furthermore even their ranks can be different. In the left figure, the subsystems {a 1 x b 1, a 2 x b 2 } and {a 3 x b 3 } determine the same face F. In the right, the 0-dimensional face F has the determining subsystems of rank 1, 2, and 3. Optimization Lab. 23rd April 2018 8 / 38
Faces Let A x = b and A x = b be two face subsystems of F : F = {x P : A x = b } = {x P : A x = b }. Then F = {x P : A x = b, A x = b }. The merged subsystem also a face subsystem of F. Definition 2.4 Maximum face subsystem The subsystem A x b obtained by merging all the face subsystems of a face F is called the maximum face subsystem of F. By definition, for any inequality not in its maximum face subsystem, F has a point satisfying it with strict inequality. Therefore, the dimension of F is the same as the dimension of the affine set A x = b : dim(f ) = n r(a ). Optimization Lab. 23rd April 2018 9 / 38
Faces Example 2.5 The maximum subsystem of the top extreme point has five inequalities, each set of three inequalities from which is also a face subsystem of it. Optimization Lab. 23rd April 2018 10 / 38
Faces Assume Ax = b has a solution. Recall Exercise 9.2 that if c T increases the rank of A, the system Ax = b, c T x = d has a solution. Otherwise, Ax = b, c T x = d has either no solution or the same solution set as Ax = b. Let F and F be the faces of P having A x b and A x b, respectively, as their maximum face subsystems. Suppose F F. Then A x = b should contain every inequality of A x = b. If, in addition, F F, then A x = b has more inequalities than A x b. And we should have r(a ) < r(a ) since, otherwise, the solution set does not change or is empty. Thus we have proved the following proposition. Proposition 2.6 Suppose two faces F and F of a polyhedron satisfy F F. Then dim(f ) < dim(f ). Optimization Lab. 23rd April 2018 11 / 38
Some typical arguments Faces 1 If a face F of P (or P itself) has at the same time the points satisfying its constraint a T x β with = and >, then F := F {a T x = β} is a face of P such that F F. 2 Consider a face F (F can be P itself) and x any point of F. Take any straight line passing through x. Suppose there is the last point z of P on the line. Then there should be a constraint a T x β which z satisfies with equality and the later points on the line will violate. If x z, we have a T x > β (since if a T x = β, the hyperplane a T x = β includes the whole line.) Therefore, from 1, we get a face F := F {a T x = β} F. Therefore, if a face F (or P ) contains no face among its proper subsets, it should include every line passing through two distinct point of it. Namely, F (or P ) is an affine set. Optimization Lab. 23rd April 2018 12 / 38