# Linear Programming Duality Homework

Here's another (I think) way to look at it. Since the dual of the dual is the primal, let's flip the question: if the primal solution is degenerate, how do we know the dual has multiple optima? (I'll give the "intuitive graphical" argument; nailing down details may or may not be easy.)

If the primal solution is degenerate, there's at least one constraint binding at the optimum such that if you relax the constraint, the optimum does not move. So the shadow price for relaxation is zero. On the other hand, tightening that constraint will cut off the current optimum. Since we're also assuming the dual solution is non-degenerate, the primal cannot have multiple optima, so cutting off the solution "hurts" the objective value by a nonzero amount ... which means the shadow price for tightening is nonzero. That gives us two different shadow prices for the same constraint, depending on which way we perturb it, so the dual must have multiple solutions.

link

answered 03 Mar '12, 11:28

Paul Rubin ♦♦
14.4k●4●12
accept rate:19%

In this part of the course, we will study some more advanced techniques for

designing approximation algorithms.

In particular, we will use linear programming duality and

semi-definite programming.

And more on geometric embeddings and use that to solve a variety of problems.

For example, Steiner forest,

facility location, maximum cut, and sparsest cut.

For all of these problems, we will design approximation algorithms and

proof theorems about them.

Minimize the function of three variables, x1, x2, x3.

For the next one, plus x2 plus 5 x3 is the objective.

And we have two constraints x1 minus x2 plus 3x3 is at least ten.

Five x1 plus 2x2 minus x3 is at least six.

Plus the non-negativity constraints.

X1, x2, and x3 cannot be strictly negative.

Let's try x1 equal to 7, x2 equal to 0,

x3 equal to 1.

Then x1-x2+3x3 equals 7- 0 + 3, that's 10.

It is greater than or equal to 10, so

the first constraint is satisfied.

five x1 + 2x2- x3 is 35,

plus 0 minus 1, that's 34.

This is greater than or equal to 6, so the second constraint is satisfied.

And of course seven, zero, and one are all greater than or equal to zero.

So, 701 is a feasible solution.

And what is its value?

Its value is seven times seven, 49 plus zero plus five, that's 54.

So, we have exhibited a particular x1, x2, x3 that has value 54.

So, the minimum possible value, when you minimize over all feasible xs,

the result must be less than or equal to 54.

And so, with this exhibit of a feasible solution,

we have proved that the optimal value is less than or equal to 54.

In other words we have a minimization problem.

How do we prove that OPT is less than something?

How do we certify an upper bound on OPT?

how do we certify that OPT is at least 10?

how do we certify that the minimum over all feasible

x's of 7x 1 plus x2 plus 5 x3 Is at least 10?

This looks more challenging, why?

Because, even if we look at the particular feasible x, or 2, or 3.

and if they're all greater than 10.

How do know that when we minimize,

the minimum over all the entire polytope, there isn't somewhere?

Some x1, x2, x3, that is feasible and has value, at least 10.

So, how can we certify that?

Let's add 2 times the first constraint plus 1 times the second constraint.

2 times x1 minus x2 plus 3x3 and

1 times 5x1 plus 2x2 + 2x3.

If you add 2 to the first constraints plus one time the second constraint

what do you get?

You get 7 x1, 2 plus 5.

Now what's the coefficient of x2 -2+0?

The coefficient on x3, 6-1, 5.

7x1 plus 5x3, must be greater than or

equal to 2*10+6, 26.

By combining the two constraints,

we derived a new constraint, 7x1 plus 5x3 is at least 26.

And now, 7x1plus x2 x5 x3 have the objective function.

If you look at each coefficient, it's greater than or

equal to the corresponding coefficient of this new constraint.

And so, the value of the objective function must be for

every feasible x greater than 7x1+5x3, which we know is at least 26.

So, have just proved that OPT is at least 26.

So, this is a way to get a lower bound for a minimization problem.

Then we'll be good.

What does it mean?

For example, take the first coefficient in the objective function 7, 7.

What is the coefficient of x1 in the new constraint?

Let's see.

We multiply the first constraint by y1, so we'll have y1 times 1.

The second constraint by y2, so we get y2 times 5.

Our coefficient is y1 + 5y2.

If 7 is great than or equal to y1 + 5y2.

The best lower bound is the tightest one.

It's the one such that this value is as large as possible.

It's the maximum value of 10 y1 plus 6 y2.

So what are we trying to do?

We're trying to find y1 and y2 that are known negative,

that satisfy these three conditions, maximizes it.

This is the dual of the initial linear program we wrote,

we started from in the example.

So, that's what we've done.

We've taken the primal LP, we've looked at the what kind of lower bonds we can get.

We wrote this as a linear program and this is the dual linear program.

Notice this -3 here should be a +3, this is a typo.

Okay, so that's duality by example.

Now, each constraint here

gives us a coefficient variable in the dual.

Constraint one gives us variable one one.

Constraint two gives us variable y2.

Similarly, each constraint in the dual corresponds to a variable in the primal.

The 7 that is here is this objective, is there in the constraint of the dual.

The 5 is here and what about these coefficients?

These are terms 10 and 6.

You find them in the objective of the dual.

And if you consider this matrix, the matrix 1 -1, 352- 1.

You find this matrix again here,

except it's the transpose of the matrix.

So with this example,

we have already a hint of the general formulation of LP duality.

We have seen on an example how you can construct the dual of a linear program.

It only remains to make this more general.