Convex Optimization - last lecture at Stanford

From Wikimization

(Difference between revisions)
Jump to: navigation, search
Current revision (13:48, 23 October 2019) (edit) (undo)
(Conclusion)
 
(15 intermediate revisions not shown.)
Line 1: Line 1:
-
I attended Prof. [http://www.stanford.edu/~boyd Stephen Boyd's] class on [http://www.stanford.edu/class/ee364a Convex Optimization] in 1999.
+
[[Image:BoydOsgood.jpg|thumb|right|731px|Stephen Boyd & Brad Osgood (right) behind Packard building (amidst construction) prior to Stanford 2009 Commencement]]
-
At that time, there was no book; just Boyd's lecture notes and figures drawn freehand.
+
In 1999, Prof. [[Boyd|Stephen Boyd's]] class on [http://www.stanford.edu/class/ee364a Convex Optimization]
 +
required no textbook; just his lecture notes and figures drawn freehand.
Boyd said there were about 100 people in the world who understood the topic.
Boyd said there were about 100 people in the world who understood the topic.
 +
At the time of his first lecture in Spring 2009, that number of people had risen to 1000 by his estimation.
-
I attended Boyd's class again in Spring 2009.
+
It was fascinating to witness evolution of this Course at Stanford over that ten year period,
-
By this time, that number of people had risen to 1000 in his estimation.
+
but more fascinating were the last five minutes of the last class in Spring 2009.
-
It was fascinating to witness evolution of his Course at Stanford over that ten year period;
+
Perhaps because the lecture was not videotaped, Boyd revealed more about his professional experiences than he may have otherwise.
-
but more fascinating were the last five minutes of the last class.
+
-
Perhaps because the lecture was not videotaped, he revealed more about his professional experiences than he may have otherwise.
+
Certainly, this was something he had not shared throughout the preceding lectures which were quite academic, in the best sense:
-
 
+
-
Certainly, this was something he had not shared throughout the preceding lectures which were quite academic in the best sense...
+
==Revelation==
==Revelation==
Some of Boyd's colleagues and contemporaries "don't believe in" Convex Optimization.
Some of Boyd's colleagues and contemporaries "don't believe in" Convex Optimization.
-
Always light in his presentation, Boyd recounted several incidents over his career in quite a humorous entertaining manner: Essentially, a colleague would ask for help solving a problem; say, in antenna design.
+
Always light in his presentation, Boyd recounted several incidents over his career in quite a humorous entertaining manner:
-
Boyd would present optimal solution to the problem, but then that colleague would reply by pointing out weakness in the solution Boyd provided.
+
Essentially, a colleague would ask for help solving a problem; say, in antenna design.
 +
Boyd would provide optimal solution to the problem, but then that colleague would reply by pointing out its weaknesses.
The colleague might say something like "''Everyone knows that those coefficients can't be negative because the antenna will wiggle uncontrollably.''"
The colleague might say something like "''Everyone knows that those coefficients can't be negative because the antenna will wiggle uncontrollably.''"
Inevitably, Boyd would respond by reminding that colleague: there was no previous mention of such a constraint.
Inevitably, Boyd would respond by reminding that colleague: there was no previous mention of such a constraint.
Line 30: Line 30:
The most difficult part of all is to express a problem well.
The most difficult part of all is to express a problem well.
-
==Background==
+
==Conclusion==
-
Stephen Boyd did not invent Convex Optimization, but he probably deserves much of the credit for its popularization in engineering:
+
Stephen Boyd did not invent Convex Optimization, but he probably deserves much of the credit for its popularization in engineering.
 +
His book with Vandenberghe <i>Convex Optimization</i> (2004) certainly made this emerging field of mathematics more accessible.
-
*He was able to interpret and distill the complicated mathematics of Convex Analysis and then present its essence in a way that is accessible to engineers.
 
- 
-
*He demonstrated applications of Convex Optimization to Control Theory and Circuit Analysis to which he made important contributions.
 
- 
-
The consequence of his efforts is to bring an obscure topic into mainstream (Convex Optimization is now a mandatory course at Stanford).
 
- 
-
He currently has the most successful and widely read book on the subject (reckoning by Amazon.com) since its release in 2004.
 
- 
-
==Conclusion==
 
If they don't believe Stephen Boyd, who then?
If they don't believe Stephen Boyd, who then?
Line 47: Line 39:
There is little room for interpretation as there may be in other disciplines (''e.g.'', Law).
There is little room for interpretation as there may be in other disciplines (''e.g.'', Law).
Mathematical results are traditionally presented within a theorem/proof paradigm.
Mathematical results are traditionally presented within a theorem/proof paradigm.
-
A proof represents culmination of labor that can span many years.
+
A proof represents culmination of labor that can span many years; indeed, Fermat's last theorem took more than 300 years.
Physicist Richard Feynman calls ''trivial'' any proven theorem, but not to diminish the result.
Physicist Richard Feynman calls ''trivial'' any proven theorem, but not to diminish the result.
So we may interpret disbelief in Convex Optimization as disbelief in the proofs; not disproof, just plain disbelief.
So we may interpret disbelief in Convex Optimization as disbelief in the proofs; not disproof, just plain disbelief.
What is disquieting about Boyd's revelation is that these particular colleagues are intelligent, successful, and learned people from academia and industry.
What is disquieting about Boyd's revelation is that these particular colleagues are intelligent, successful, and learned people from academia and industry.
- 
- 
-
--[[User:Kwokwah|Kwokwah]] 18:57, 11 August 2009 (PDT)
 

Current revision

Stephen Boyd & Brad Osgood (right) behind Packard building (amidst construction) prior to Stanford 2009 Commencement
Stephen Boyd & Brad Osgood (right) behind Packard building (amidst construction) prior to Stanford 2009 Commencement

In 1999, Prof. Stephen Boyd's class on Convex Optimization required no textbook; just his lecture notes and figures drawn freehand. Boyd said there were about 100 people in the world who understood the topic. At the time of his first lecture in Spring 2009, that number of people had risen to 1000 by his estimation.

It was fascinating to witness evolution of this Course at Stanford over that ten year period, but more fascinating were the last five minutes of the last class in Spring 2009.

Perhaps because the lecture was not videotaped, Boyd revealed more about his professional experiences than he may have otherwise.

Certainly, this was something he had not shared throughout the preceding lectures which were quite academic, in the best sense:

Revelation

Some of Boyd's colleagues and contemporaries "don't believe in" Convex Optimization.

Always light in his presentation, Boyd recounted several incidents over his career in quite a humorous entertaining manner: Essentially, a colleague would ask for help solving a problem; say, in antenna design. Boyd would provide optimal solution to the problem, but then that colleague would reply by pointing out its weaknesses. The colleague might say something like "Everyone knows that those coefficients can't be negative because the antenna will wiggle uncontrollably." Inevitably, Boyd would respond by reminding that colleague: there was no previous mention of such a constraint. The new constraint would be accounted for, the problem solved again, and then another round of the same would ensue.

Another colleague proposed that in some circumstances "One does not want an optimal solution";  implying, optimal solutions are bad in some sense.

Yet another esteemed colleague posited an optimal solution provided by Boyd as proof that "Convex Optimization doesn't work."

Boyd ended the lecture with the moral of his recounting (which I paraphrase): If constraints are incomplete, then an optimal solution cannot be right. The most difficult part of all is to express a problem well.

Conclusion

Stephen Boyd did not invent Convex Optimization, but he probably deserves much of the credit for its popularization in engineering. His book with Vandenberghe Convex Optimization (2004) certainly made this emerging field of mathematics more accessible.

If they don't believe Stephen Boyd, who then?

The beauty of mathematics is that either it is right or it is wrong; e.g., the left side of an equation equals the right side. There is little room for interpretation as there may be in other disciplines (e.g., Law). Mathematical results are traditionally presented within a theorem/proof paradigm. A proof represents culmination of labor that can span many years; indeed, Fermat's last theorem took more than 300 years. Physicist Richard Feynman calls trivial any proven theorem, but not to diminish the result.

So we may interpret disbelief in Convex Optimization as disbelief in the proofs; not disproof, just plain disbelief. What is disquieting about Boyd's revelation is that these particular colleagues are intelligent, successful, and learned people from academia and industry.

Personal tools